* refactor: dedupe edge api json response assembly
* refactor: expand jsonResponse helper to all edge functions
Roll out jsonResponse() from _json-response.js to 16 files (14 handlers
+ 2 shared helpers), eliminating 55 instances of the
new Response(JSON.stringify(...)) boilerplate.
Only exception: health.js uses JSON.stringify(body, null, indent) for
pretty-print mode, which is incompatible with the helper signature.
Replaced local jsonResponse/json() definitions in contact.js,
register-interest.js, and cache-purge.js with the shared import.
shippingRates was in SLOW_KEYS (2h CDN cache). After seed populates new
freight indices (SCFI, CCFI, BDI), CF served stale bootstrap with only
2 FRED indices for up to 2.5h. Moving to FAST_KEYS (10min CDN TTL)
ensures new data appears promptly.
Also update BOOTSTRAP_TIERS in cache-keys.ts to match.
* feat(trade): add US Treasury customs revenue to Trade Policy panel
US customs duties revenue spiked 4-5x under Trump tariffs (from
$7B/month to $27-31B/month) but the WTO tariff data only goes to
2024. Adds Treasury MTS data showing monthly customs revenue.
- Add GetCustomsRevenue RPC (proto, handler, cache tier)
- Add Treasury fetch to seed-supply-chain-trade.mjs (free API, no key)
- Add Revenue tab to TradePolicyPanel with FYTD YoY comparison
- Fix WTO gate: per-tab gating so Revenue works without WTO key
- Wire bootstrap hydration, health, seed-health tracking
* test(trade): add customs revenue feature tests
22 structural tests covering:
- Handler: raw key mode, empty-cache behavior, correct Redis key
- Seed: Treasury API URL, classification filter, timeout, row
validation, amount conversion, sort order, seed-meta naming
- Panel: WTO gate fix (per-tab not panel-wide), revenue tab
defaults when WTO key missing, dynamic FYTD comparison
- Client: no WTO feature gate, bootstrap hydration, type exports
* fix(trade): align FYTD comparison by fiscal month count
Prior FY comparison was filtering by calendar month, which compared
5 months of FY2026 (Oct-Feb) against only 2 months of FY2025
(Jan-Feb), inflating the YoY percentage. Now takes the first N
months of the prior FY matching the current FY month count.
* fix(trade): register treasury_revenue DataSourceId and localize revenue tab
- Add treasury_revenue to DataSourceId union type so freshness
tracking actually works (was silently ignored)
- Register in data-freshness.ts source config + gap messages
- Add i18n keys: revenue tab label, empty state, unavailable banner
- Update infoTooltip to include Revenue tab description
* fix(trade): complete revenue tab localization
Use t() for all remaining hardcoded strings: footer source labels,
FYTD summary headline, prior-year comparison, and table column
headers. Wire the fytdLabel/vsPriorFy keys that were added but
not used.
* fix(test): update revenue source assertion for localized string
* feat(forecast): add AI Forecasts prediction module (Pro-tier)
MiroFish-inspired prediction engine that generates structured forecasts
across 6 domains (conflict, market, supply chain, political, military,
infrastructure) using existing WorldMonitor data streams.
- Proto definitions for ForecastService with GetForecasts RPC
- Dedicated seed script (seed-forecasts.mjs) with 6 domain detectors,
cross-domain cascade resolver, prediction market calibration, and
trend detection via prior snapshot comparison
- Premium-gated RPC handler (PREMIUM_RPC_PATHS enforcement)
- Lazy-loaded ForecastPanel with domain filters, probability bars,
trend arrows, signal evidence, and cascade links
- Health monitoring integration (seed-meta freshness tracking)
- Refresh scheduler with API key guard
* test(forecast): add 47 unit tests for forecast detectors and utilities
Covers forecastId, normalize, resolveCascades, calibrateWithMarkets,
computeTrends, and smoke tests for all 6 domain detectors. Exports
testable functions from seed script with direct-run guard.
* fix(forecast): domain mismatch 'infra' vs 'infrastructure', add panel category
- Seed script used 'infra' but ForecastPanel filtered on 'infrastructure',
causing Infra tab to show zero results
- Added 'forecast' to intelligence category in PANEL_CATEGORY_MAP
* fix(forecast): move CSS to one-time injection, improve type safety
- P2: Move style block from setContent to one-time document.head injection
to prevent CSS accumulation on repeated renders
- P3: Replace +toFixed(3) with Math.round for readability in seed script
- P3: Use Forecast type instead of any[] in RPC handler filter
* fix(forecast): handle sebuf proto data shapes from Redis
Detectors now normalize CII scores from server-side proto format
(combinedScore, TREND_DIRECTION_RISING, region) to uniform shape.
Outage severity handles proto enum format (SEVERITY_LEVEL_HIGH).
Added confidence floor of 0.3 for single-source predictions.
Verified against live Redis: 2 predictions generated (Iran infra
shutdown, IL political instability).
* feat(forecast): unlock AI Forecasts on web, lock desktop only (trial)
- Remove forecast RPC from PREMIUM_RPC_PATHS (web access is free)
- Panel locked on desktop only (same as oref-sirens/telegram-intel)
- Remove API key guards from data-loader and refresh scheduler
- Web users get full access during trial period
* chore: regenerate proto types with make generate
Re-ran make generate after rebasing on main. Plugin v0.7.0 dropped
@ts-nocheck from output, added it back to all 50 generated files.
Fixed 4 type errors from proto codegen changes:
- MarketSource enum -> string union type
- TemporalAnomalyProto -> TemporalAnomaly rename
- webcam lastUpdated number -> string
* chore: add proto freshness check to pre-push hook
Runs make generate before push and compares checksums of generated files.
If proto types are stale, blocks push with instructions to regenerate.
Skips gracefully if buf CLI is not installed.
* fix(forecast): use chokepoints v4 key, include ciiContribution in unrest
- P1: Switch chokepoints input from stale v2 to active v4 Redis key,
matching bootstrap.js and cache-keys.ts
- P2: Add ciiContribution to unrest component fallback chain in
normalizeCiiEntry so political detector reads the correct sebuf field
* feat(forecast): Phase 2 LLM scenario enrichment + confidence model
MiroFish-inspired enhancements:
- LLM scenario narratives via Groq/OpenRouter (narrative-only, no numeric
adjustment). Evidence-grounded prompts with mandatory signal citation
and few-shot examples from MiroFish's SECTION_SYSTEM_PROMPT_TEMPLATE.
- Top-4 predictions batched into single LLM call for cost efficiency.
- News context from newsInsights attached to all predictions for LLM
prompt grounding (NOT in signals, cannot affect confidence).
- Deterministic confidence model: source diversity via SIGNAL_TO_SOURCE
mapping (deduplicates cii+cii_delta, theater+indicators) + calibration
agreement from prediction market drift. Floor 0.2, ceiling 1.0.
- Output validation: rejects scenarios without signal references.
- Truncated JSON repair for small model output.
- Structured JSON logging for LLM calls.
- Redis cache for LLM scenarios (1h TTL).
- 23 new tests (70 total), all passing.
- Live-tested: OpenRouter gemini-2.5-flash produces evidence-grounded
scenario narratives from real WorldMonitor data.
* feat(forecast): Phase 3 multi-perspective scenarios, projections, data-driven cascades
MiroFish-inspired enhancements:
- Multi-perspective LLM analysis: top-2 predictions get strategic,
regional, and contrarian viewpoints via combined LLM call
- Probability projections: domain-specific decay curves (h24/d7/d30)
anchored to timeHorizon so probability equals projections[timeHorizon]
- Data-driven cascade rules: moved from hardcoded array to JSON config
(scripts/data/cascade-rules.json) with schema validation, named
predicate evaluators, unknown key rejection, and fallback to defaults
- 4 new cascade paths: infrastructure->supply_chain, infrastructure->market
(both requiresSeverity:total), conflict->political, political->market
- Proto: added Perspectives and Projections messages to Forecast
- ForecastPanel: renders projections row and conditional perspectives toggle
- 89 tests (19 new), all passing
- Live-tested: OpenRouter produces perspectives from real data
* feat(forecast): Phase 4 data utilization + entity graph
Fixes data gaps that prevented 4 of 6 detectors from firing:
- Input normalizers: chokepoint v4 shape + GPS hexes-to-zones mapping
- Chokepoint warm-ping (production-only, requires WM_API_BASE_URL)
- Lowered CII conflict threshold from 70 to 60, gated on level=high|critical
4 new standalone detectors:
- UCDP conflict zones (10+ events per country)
- Cyber threat concentration (5+ threats per country)
- GPS jamming in maritime shipping zones (5 regions)
- Prediction markets as signals (60-90% probability markets)
Entity-relationship graph (file-based, 38 nodes):
- Countries, theaters, commodities, chokepoints, alliances
- Alias table resolves both ISO codes and display names
- Graph cascade discovery links predictions across entities
Result: 51 predictions (up from 1-2), spanning conflict, infrastructure,
and supply chain domains. 112 tests, all passing.
* fix(forecast): redis cache format, signal source mapping, type safety
Fresh-eyes audit fixes:
- BUG: redisSet used wrong Upstash API format (POST body with {value,ex}
instead of command array ['SET',key,value,'EX',ttl]). LLM cache writes
were silently failing, causing fresh LLM calls every run.
- BUG: prediction_market signal type missing from SIGNAL_TO_SOURCE,
inflating confidence for market-derived predictions.
- CLEANUP: Remove unnecessary (f as any) casts in ForecastPanel since
generated Forecast type already has projections/perspectives fields.
- CLEANUP: Bump health maxStaleMin from 60 to 90 to avoid false STALE
alerts when LLM calls add latency to seed runs.
* feat(forecast): headline-entity matching with news corroboration signals
Uses entity graph aliases to match headlines to predictions by
country/theater (excludes commodity/infrastructure nodes to prevent
false positives). Predictions with matching headlines get a
news_corroboration signal visible in the panel.
Also fixes buildUserPrompt to merge unique headlines from ALL
predictions in the LLM batch (was only reading preds[0].newsContext).
Live-tested: 13 of 51 predictions now have corroborating headlines
(Iran, Israel, Syria, Ukraine, etc). 116 tests, all passing.
* feat(forecast): add country-codes.json for headline-entity matching
56 countries with ISO codes, full names, and scoring keywords (extracted
from src/config/countries.ts + UCDP-relevant additions). Used by
attachNewsContext for richer headline matching via getSearchTermsForRegion
which combines country-codes + entity graph + keyword aliases.
14/57 predictions now have news corroboration (limited by headline
coverage, not matching quality: only 8 headlines currently available).
* feat(forecast): read 300 headlines from news digest instead of 8
Read news:digest:v1:full:en (300 headlines across 16 categories) instead
of just news:insights:v1 topStories (8 headlines). Fallback to topStories
if digest is unavailable.
Result: news corroboration jumped from 25% to 64% (38/59 predictions).
* fix(forecast): handle parenthetical country names in headline matching
Strip suffixes like '(Zaire)', '(Burma)', '(Soviet Union)' from UCDP
region names before matching against country-codes.json. Also use
includes() for reverse name lookup to catch partial matches.
Corroboration: 64% -> 69% (41/59). Remaining 18 unmatched are countries
with no current English-language news coverage.
* fix(forecast): cache validated LLM output, add digest test, log cache errors
Fresh-eyes audit fixes:
- Combined LLM cache now stores only validated items (was caching raw
unvalidated output, serving potentially invalid scenarios on cache hit)
- redisSet logs warnings on failure (was silently swallowing all errors)
- Added digest-based test for attachNewsContext (primary path was untested)
- Fixed test arity: attachNewsContext(preds, news, digest) with 3 params
* fix(forecast): remove dead confidenceFromSources, reduce warm-ping timeout
- P2: Remove confidenceFromSources (dead code, computeConfidence overwrites
all initial confidence values). Inline the formula in original detectors.
- P3: Reduce warm-ping timeout from 30s to 15s (non-critical step)
- P3: Add trial status comment on forecast panel config
* fix(forecast): resolve ISO codes to country names, fix market detector, safe pre-push
P1 fixes from code review:
- CII ISO codes (IL, IR) now resolved to full country names (Israel, Iran)
via country-codes.json. Prevents substring false positives (IL matching
Chile) in event correlation. Uses word-boundary regex for matching.
- Market detector CII-to-theater mapping now uses entity graph traversal
instead of broken theater-name substring matching. Iran correctly maps
to Middle East theater via graph links.
- Pre-push hook no longer runs destructive git checkout on proto freshness
failure. Reports mismatch and exits without modifying worktree.
* feat(forecast): add structured scenario pipeline and trace export
* fix(forecast): hydrate bootstrap and trim generated drift
* fix(forecast): keep required supply-chain contract updates
* fix(ci): add forecasts to cache-keys registry and regenerate proto
Add forecasts entry to BOOTSTRAP_CACHE_KEYS and BOOTSTRAP_TIERS in
cache-keys.ts to match api/bootstrap.js. Regenerate SupplyChain proto
to fix duplicate TransitDayCount and add riskSummary/riskReportAction.
* fix(data): restore bootstrap and cache test coverage
* fix: resolve linting and test failures
- Remove dead writeSeedMeta/estimateRecordCount functions from redis.ts
(intentionally removed from cachedFetchJson; seed-meta now written
only by explicit seed flows, not generic cache reads)
- Fix globe dayNight test to match actual code (forces dayNight: false
+ hideLayerToggle, not catalog-based exclusion)
- Fix country-geometry test mock URL from CDN to /data/countries.geojson
(source changed to use local bundled file)
* fix(lint): remove duplicate llm-health key in redis-caching test
Duplicate object key '../../../_shared/llm-health' caused the stub
to be overwritten by the real module. Removed the second entry so
the test correctly uses the stub.
* feat(advisories): gold standard migration for security advisories
Move security advisories from client-side RSS fetching (24 feeds per
page load) to Railway cron seed with Redis-read-only Vercel handler.
- Add seed script fetching via relay RSS proxy with domain allowlist
- Add ListSecurityAdvisories proto, handler, and RPC cache tier
- Add bootstrap hydration key for instant page load
- Rewrite client service: bootstrap -> RPC fallback, no browser RSS
- Wire health.js, seed-health.js, and dataSize tracking
* fix(advisories): empty RPC returns ok:true, use full country map
P1 fixes from Codex review:
- Return ok:true for empty-but-successful RPC responses so the panel
clears to empty instead of stuck loading on cold environments
- Replace 50-entry hardcoded country map with 251-entry shared config
generated from the project GeoJSON + aliases, matching coverage of
the old client-side nameToCountryCode matcher
* fix(advisories): add Cote d'Ivoire and other missing country aliases
Adds 14 missing aliases including "cote d ivoire" (US State Dept
title format), common article-prefixed names (the Bahamas, the
Gambia), and alternative official names (Czechia, Eswatini, Cabo
Verde, Timor-Leste).
* fix(proto): inject @ts-nocheck via Makefile generate target
buf generate does not emit @ts-nocheck, but tsc strict mode rejects
the generated code. Adding a post-generation sed step in the Makefile
ensures both CI proto-freshness (make generate + diff) and CI
typecheck (tsc --noEmit) pass consistently.
* feat(correlation): server-side correlation engine seed + bootstrap hydration
Move correlation card computation from client-side (per-browser, 10-30s delay)
to server-side (Railway cron, instant via bootstrap). Seed script reads 8 Redis
keys, runs 4 adapter signal collectors (military, escalation, economic, disaster),
clusters/scores/generates cards, writes to Redis with 10min TTL.
- New: scripts/seed-correlation.mjs (pure JS port of correlation engine)
- bootstrap.js: add correlationCards to FAST_KEYS tier
- health.js + seed-health.js: register for monitoring (maxStaleMin: 15)
- CorrelationPanel: consume bootstrap on construction, show "Analyzing..." only
after live engine has run (not for bootstrap-only cards)
- _seed-utils.mjs: support opts.recordCount override (function or number)
* fix(correlation): stale timestamp fallback + coordinate-based country resolution
P1: news stories lacked per-story pubDate, causing Date.now() fallback on
every seed run. Now _clustering.mjs propagates pubDate through to
enrichedStories, and seed-correlation reads s.pubDate then generatedAt.
P2: normalizeToCode dropped signals with unparseable country names.
Added centroid-based coordinate fallback (haversine nearest-match within
800km) matching the live engine's getCountryAtCoordinates behavior.
* fix(correlation): add 11 missing country centroids to coordinate fallback
CI, CR, CV, CY, GA, IS, LA, SZ, TL, TT, XK were in the normalization
maps but missing from COUNTRY_CENTROIDS, causing coordinate-only signals
in those countries to be misclassified or dropped during bootstrap.
* fix(correlation): align protest/outage field names with actual Redis schema
Codex review P1 findings: seed-correlation read wrong field names from
Redis data.
Protests (unrest:events:v1): p.time -> p.occurredAt, p.lat/lon ->
p.location.latitude/longitude, severity enum SEVERITY_LEVEL_* mapping.
Outages (infra:outages:v1): o.pubDate -> o.detectedAt, o.lat/lon ->
o.location.latitude/longitude, severity enum OUTAGE_SEVERITY_* mapping.
Both escalation and disaster adapters updated. Old field names kept as
fallbacks for data shape compatibility.
* feat(supply-chain): replace S&P Global with 3 free maritime data sources
Replace expensive S&P Global Maritime API with IMF PortWatch (vessel transit
counts), CorridorRisk (risk intelligence), and AISStream chokepoint crossing
counter. All external API calls run on Railway relay, Vercel reads Redis only.
- Add 4 new chokepoints (10 total): Cape of Good Hope, Gibraltar, Bosphorus, Dardanelles
- Add TransitSummary proto (field 14) with today counts, WoW%, 180d history, risk context
- Add D3 multi-line chart (tanker vs cargo) with expandable chokepoint cards
- Add crossing detection with enter+dwell+exit semantics, 30min cooldown, 5min min dwell
- Add PortWatch seed loop (6h), CorridorRisk seed loop (1h), transit seed loop (10min)
- Add canonical chokepoint ID map for cross-source name resolution
- 177 tests passing across 6 test files
* fix(supply-chain): address P2 review findings
- Discard partial PortWatch pagination results on mid-page failure (prevents
truncated history with wrong WoW numbers cached for 6h)
- Rename "Transit today" to "24h" label (rolling 24h window, not calendar day)
- Fix chart label from "30d" to "180d" (matches actual PortWatch query range)
- Add 30s initial seed for chokepoint transits on relay cold start (prevents
10min gap of zero transit data)
* feat(supply-chain): swap D3 chart for TradingView lightweight-charts
Replace hand-rolled D3 SVG transit chart with lightweight-charts v5 canvas
rendering for Bloomberg-quality time-series visualization.
- Add TransitChart helper class with mount/destroy lifecycle, theme listener,
and autoSize support
- Use MutationObserver (not rAF) to mount chart after setContent debounce
- Clean up chart on tab switch, collapse, and re-render (no orphaned canvases)
- Respond to theme-changed events via chart.applyOptions()
- D3 stays for other 5 components (ProgressCharts, RenewableEnergy, etc.)
* feat(supply-chain): add geo coords and trade routes for 4 new chokepoints
Cherry-pick from PR #1511: Cape of Good Hope, Gibraltar, Bosphorus, and
Dardanelles map-layer coordinates and trade route definitions.
* fix(supply-chain): health.js v2->v4 key + double cache TTLs for missed seeds
- health.js chokepoints key was still v2, now v4 (matches handler + bootstrap)
- PortWatch TTL: 21600s (6h) -> 43200s (12h), seed interval stays 6h
- CorridorRisk TTL: 3600s (1h) -> 7200s (2h), seed interval stays 1h
- Ensures one missed seed run doesn't expire the key and cause empty data
* feat: seed GDELT intelligence topics to Redis with bootstrap hydration
Add standalone seed script that pre-populates all 6 Live Intelligence
topics (military, cyber, nuclear, sanctions, intelligence, maritime)
from the GDELT Doc API into Redis. Frontend consumes bootstrap data
lazily via the service layer, falling back to RPC if unavailable.
- scripts/seed-gdelt-intel.mjs: new seed script with per-topic 429 retry
- api/bootstrap.js: register gdeltIntel in FAST_KEYS
- api/health.js: register in BOOTSTRAP_KEYS + SEED_META + dataSize
- api/seed-health.js: register in SEED_DOMAINS
- scripts/_seed-utils.mjs: add topics to recordCount detection
- src/services/gdelt-intel.ts: lazy bootstrap consumption in service layer
* fix(seed): align staleness thresholds and strengthen GDELT validation
- seed-health intervalMin 30→60 so staleness (120min) matches health.js maxStaleMin
- validate requires ≥3/6 topics populated (not just military)
- recordCount sums articles across topics instead of reporting topic count
* fix(tech-events): prevent partial fetch results from being cached
Techmeme ICS and dev.events RSS fetches on Vercel edge can partially
fail (timeout, truncation), returning only 1 event instead of 20+.
The handler cached this partial result for 6 hours, causing the Tech
Events panel to show empty.
- Add 8s AbortSignal.timeout on both external fetches
- Require minimum 5 events before caching (at least curated count)
* fix(tech-events): remove MIN_EVENTS threshold and add diagnostic logging
The MIN_EVENTS=5 threshold caused empty results when both external
sources fail on Vercel edge (only 4 curated events available). Now
any events > 0 are cached. Added detailed logging to diagnose why
Techmeme ICS and dev.events RSS fetches fail on Vercel edge.
Also removed past STEP Dubai 2026 event.
* fix(tech-events): route fetches through Railway relay when direct fails
Vercel edge functions cannot reliably reach Techmeme ICS and dev.events
RSS (datacenter IP blocking). Added fetchTextWithRelay() that tries
direct fetch first, then falls back to Railway relay proxy (/rss endpoint)
which fetches from a different IP. Same pattern used by news feed digest
and other handlers that hit blocked external sources.
* feat(tech-events): gold standard pipeline with Railway seed + bootstrap hydration
Full data pipeline overhaul to match project conventions:
- Add tech events seed loop to ais-relay.cjs: fetches Techmeme ICS +
dev.events RSS every 6h from Railway (avoids Vercel IP blocking),
parses both sources, merges with curated fallback events, writes to
Redis (data key + bootstrap key + seed-meta)
- Register in api/bootstrap.js BOOTSTRAP_CACHE_KEYS (SLOW tier)
- Register in api/health.js BOOTSTRAP_KEYS + SEED_META (420min stale)
- Restructure RPC handler: reads from single broad Redis key (populated
by seed), applies geocoding + filtering in-memory per request params.
Fallback fetcher only runs on cold start before first seed
- TechEventsPanel: check getHydratedData('techEvents') from bootstrap
before falling back to RPC call
- data-loader: use hydrated bootstrap data for map layer, RPC fallback
Move theaterPosture from SLOW (2h CDN) to FAST tier (20min/10min after
PR #1314) so military posture data stays fresh. Increase risk scores
breaker TTL to 30min to match health.js maxStaleMin, and reduce
localStorage staleness from 24h to 1h to prevent stale risk data in UI.
CDN-Cache-Control for fast tier was 20min + 5min stale-while-revalidate,
but insights-loader rejects data older than 15min. This caused frequent
fallback to the slow 4-step client-side AI pipeline. Align CDN TTL with
origin Cache-Control (10min + 2min SWR) so CF revalidates before the
freshness threshold.
* fix: eliminate frontend external API calls, enforce gold standard pattern
- Polymarket: remove browser fan-out (536→105 lines), bootstrap → RPC only
- USASpending: remove direct API calls, read from bootstrap hydration
- NWS Weather: remove direct API calls, read from bootstrap hydration
- Nominatim: proxy through api/reverse-geocode.js with Redis cache + SSRF clamping
- Add seed scripts for weather alerts (15min) and spending (60min)
- Wire both seed loops into ais-relay.cjs
- Register weatherAlerts + spending in bootstrap.js and health.js
- Add 4 missing standalone keys to health.js (cyberThreatsRpc, militaryBases, temporalAnomalies, displacement)
* fix: resolve reload regressions and null-cache poisoning from #1217
- Weather/Spending: fall back to `/api/bootstrap?keys=` on scheduled
reloads after the one-shot `getHydratedData()` is consumed
- Prediction: add client-side bootstrap filter for country markets
when RPC fails (server skips bootstrap for query-based requests)
- Reverse-geocode: restore abort/timeout guard so transient network
errors don't permanently poison the in-memory cache
* perf(baseline): move temporal baseline for news+fires to server-side
Every browser client was calling record-baseline-snapshot (POST) +
get-temporal-baseline (GET) on every data refresh from 5 call sites.
With N concurrent users this created N identical writes and ~5N reads
per cycle — causing 429 rate limiting and statistically biased baselines.
Phase 1 moves news and satellite_fires to server-side computation:
- New ListTemporalAnomalies RPC reads counts from existing Redis keys
(news:insights:v1, wildfire:fires:v1), computes anomalies against v2
baselines, applies Welford update (1 sample/cycle), caches 15min
- Bootstrap hydration delivers anomalies on page load (zero extra calls)
- Client refreshes via RPC every 10min (1 cached call vs 5N before)
- Remaining 3 types (military_flights, vessels, ais_gaps) stay client-side
- Owner-guarded distributed lock prevents concurrent computation
- All reads/writes use prefix-aware getCachedJson/setCachedJson
Expected ~60% reduction in baseline-related Vercel invocations.
* fix(temporal): per-invocation lock owner and immediate refresh on cold cache
P1: When bootstrap has no temporal anomaly data (cold cache, expired
snapshot, fresh deploy), fire refreshTemporalBaseline() immediately
instead of waiting up to 10 minutes for the scheduled refresh.
P2: Generate lockOwner per invocation via makeLockOwner() instead of
once at module load. Prevents warm edge isolates from cross-releasing
each other's locks when one invocation outlives the 30s TTL.
* fix(temporal): use TTL-only lock instead of TOCTOU GET-then-DEL release
The non-atomic GET→check-owner→DEL release was vulnerable to a race
where the TTL expires between GET and DEL, allowing a new lock holder
to be evicted. Simplify by relying solely on the 30s TTL for lock
expiry — the computation completes well within that window.
* fix(economic): seed all WB indicators on Railway, never call WB API from frontend
Extends seed-wb-indicators.mjs to pre-compute progress data (4 indicators)
and renewable energy data (EG.ELC.RNEW.ZS) alongside tech readiness rankings.
Frontend callers (progress-data.ts, renewable-energy-data.ts, getTechReadinessRankings,
getCountryComparison) now read exclusively from bootstrap/Redis seed data.
Zero Vercel Edge → World Bank API calls remain.
* fix: address code review findings (P1+P2)
- Fix triple JSON.parse in seed verification (P1)
- Graceful fallback for renewable data fetch failure (P2)
- Use Map lookup instead of Array.find in progress-data (P2)
- Update regression test for bootstrap-only getTechReadinessRankings (P2)
Register 6 new seeds in bootstrap.js (crypto, gulf, stablecoin, unrest,
iran, ucdp) and wire getHydratedData() in 7 service files. Also adds
hydration for 2 previously-registered keys (cyberThreats, flightDelays)
that had no frontend consumer. Syncs cache-keys.ts with bootstrap.js
for test parity.
Cyber hydration correctly maps through toCyberThreat() to convert proto
enum strings to friendly types.
* perf: reduce Vercel data transfer costs with CDN optimization
- Increase polling intervals (markets 8→12min, feeds 15→20min, crypto 8→12min)
- Increase background tab hiddenMultiplier from 10→30 (polls 3x less when hidden)
- Double CDN s-maxage TTLs across all cache tiers in gateway
- Add CDN-Cache-Control header for Cloudflare-specific longer edge caching
- Add ETag generation + 304 Not Modified support in gateway (zero-byte revalidation)
- Add CDN-Cache-Control to bootstrap endpoint
- Add explicit SPA rewrite rule in vercel.json for CF proxy compatibility
- Add Cache-Control headers for /map-styles/, /data/, /textures/ static paths
* fix: prevent CF from caching SPA HTML + reduce Polymarket bandwidth 95%
- vercel.json: apply no-cache headers to ALL SPA routes (same regex as
rewrite rule), not just / and /index.html — prevents CF proxy from
caching stale HTML that references old content-hashed bundle filenames
- Polymarket: add server-side aggregation via Railway seed script that
fetches all tags once and writes to Redis, eliminating 11-request
fan-out per user per poll cycle
- Bootstrap: add predictions to hydration keys for zero-cost page load
- RPC handler: read Railway-seeded bootstrap key before falling back to
live Gamma API fetch
- Client: 3-strategy waterfall (bootstrap → RPC → fan-out fallback)
* feat: Implement comprehensive aviation monitoring service with flight search, status, news, and tracking.
* feat: Introduce Airline Intelligence Panel with aviation data tabs, map components, and localization.
* feat: Implement DeckGL-based map for advanced visualization, D3/SVG fallback, i18n support, and aircraft tracking.
* Update server/worldmonitor/aviation/v1/get-carrier-ops.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/search-flight-prices.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/track-aircraft.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/get-airport-ops-summary.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update proto/worldmonitor/aviation/v1/position_sample.proto
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/list-airport-flights.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update proto/worldmonitor/aviation/v1/price_quote.proto
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: Add server-side endpoints for aviation news and aircraft tracking, and introduce a new DeckGLMap component for map visualization.
* Update server/worldmonitor/aviation/v1/list-airport-flights.ts
The cache key for listAirportFlights excludes limit, but the upstream fetch/simulated generator uses limit to determine how many flights to return. If the first request within TTL uses a small limit, larger subsequent requests will be incorrectly capped until cache expiry. Include limit (or a normalized bucket/max) in cacheKey, or always fetch/cache a fixed max then slice per request.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/get-flight-status.ts
getFlightStatus accepts origin, but cacheKey does not include it. This can serve cached results from an origin-less query to an origin-filtered query (or vice versa). Add origin (normalized) to the cache key or apply filtering after fetch to ensure cache correctness.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: Implement DeckGL map for advanced visualization and new aviation data services.
* fix(aviation): prevent cache poisoning and keyboard shortcut in inputs
- get-carrier-ops: move minFlights filter post-cache to avoid cache
fragmentation (different callers sharing cached full result)
- AviationCommandBar: guard Ctrl+J shortcut so it does not fire when
focus is inside an INPUT or TEXTAREA element
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: introduce AviationCommandBar component for parsing user commands, fetching aviation data, and displaying results.
* feat: Implement aircraft tracking service with OpenSky and simulated data sources.
* feat: introduce DeckGLMap component for WebGL-accelerated map visualizations using deck.gl and maplibre-gl.
* fix(aviation): address code review findings for PR #907
Proto: add missing (sebuf.http.query) annotations on all GET request
fields across 6 proto files; add currency/market fields to
SearchFlightPricesRequest.
Server: add parseStringArray to aviation _shared.ts and apply to
get-airport-ops-summary, get-carrier-ops, list-aviation-news handlers
to prevent crash on comma-separated query params; remove leaked API
token from URL params in travelpayouts_data; fix identical simulated
flight statuses in list-airport-flights; remove unused endDate var;
normalize cache key entity casing in list-aviation-news.
Client: refactor AirlineIntelPanel to extend Panel base class and
register in DEFAULT_PANELS for full/tech/finance variants; fix
AviationCommandBar reference leak with proper destroy() cleanup in
panel-layout; rename priceUsd→priceAmount in display type and all
usages; change auto-refresh to call refresh() instead of loadOps().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: introduce aviation command bar component with aircraft tracking and flight information services.
* feat: Add `AirlineIntelPanel` component for displaying airline operations, flights, carriers, tracking, news, and prices in a tabbed interface.
* feat: Add endpoints for listing airport flights and fetching aviation news.
* Update proto/worldmonitor/aviation/v1/search_flight_prices.proto
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: Add server endpoint for listing airport flights and client-side MapPopup types and utilities.
* feat: Introduce MapPopup component with support for various data types and responsive positioning for map features.
* feat: Add initial English localization file (en.json).
* fix(aviation): address PR review findings across aviation stack
- Add User-Agent header to Travelpayouts provider (server convention)
- Use URLSearchParams for API keys instead of raw URL interpolation
- Add input length validation on flightNumber (max 10 chars)
- Replace regex XML parsing with fast-xml-parser in aviation news
- Fix (f as any)._airport type escape with typed Map<FI, string>
- Extract DEFAULT_WATCHED_AIRPORTS constant from hardcoded arrays
- Use event delegation for AirlineIntelPanel price search listener
- Add bootstrap hydration key for flight delays
- Bump OpenSky cache TTL to 120s (anonymous tier rate limit)
- Match DeckGLMap aircraft poll interval to server cache (120s)
- Fix GeoJSON polygon winding order (shoelace check + auto-reversal)
* docs: add aviation env vars to .env.example
AVIATIONSTACK_API, ICAO_API_KEY, TRAVELPAYOUTS_API_TOKEN
* feat: Add aviation news listing API and introduce shared RSS allowed domains.
* fix: add trailing newline to rss-allowed-domains.json, remove unused ringIsClockwise
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
* fix: add circuit breaker + bootstrap to CII risk scores
Same pattern as theater posture (#948): replace fragile in-memory cache
+ manual persistent-cache with circuit breaker (SWR, IndexedDB, cooldown)
and bootstrap hydration. Eliminates learning-mode delay on cold start
and survives RPC failures without blanking the panel.
* fix: add localStorage sync prime for CII risk scores
getCachedScores() is called synchronously by country-intel.ts as a
fallback during learning mode. Without localStorage priming, the
breaker's async IndexedDB hydration hasn't run yet and returns null.
- Add shape validator (isValidCiiEntry) for untrusted localStorage data
- Add loadFromStorage/saveToStorage with 24h staleness ceiling
- Prime breaker synchronously at module load from localStorage
- Skip priming for empty cii arrays to avoid cached-empty trap
- Save to localStorage on both bootstrap and RPC success paths
* feat: Railway CII seed + bootstrap hydration for instant panel render
- Add 8-source CII seed to Railway (ACLED, UCDP, outages, climate, cyber, fires, GPS, Iran strikes)
- Neuter Vercel handler to read-only (returns Railway-seeded cache, never recomputes)
- Register riskScores in bootstrap FAST tier for CDN-cached delivery
- Add early CII hydration in data-loader before intelligence signals
- Add CIIPanel.renderFromCached() for instant render from bootstrap data
- Refactor cached-risk-scores.ts: circuit breaker + localStorage sync prime + bootstrap hydration
- Progressive enhancement: cached render → full 18-source local recompute (no spinner)
* fix: remove duplicate riskScores key in BOOTSTRAP_TIERS after merge
* feat: move EONET/GDACS to server-side with Redis caching and bootstrap hydration
Browser-direct fetches to eonet.gsfc.nasa.gov and gdacs.org caused CORS
errors and had no server-side caching. This moves both to the standard
Vercel edge → cachedFetchJson → Redis → bootstrap hydration pattern.
- Add proto definitions for NaturalService with ListNaturalEvents RPC
- Create server handler merging EONET + GDACS with 30min Redis TTL
- Add Vercel edge function at /api/natural/v1/list-natural-events
- Register naturalEvents in bootstrap SLOW_KEYS for CDN hydration
- Replace browser-direct fetches with RPC client + circuit breaker
- Delete src/services/gdacs.ts (logic moved server-side)
* fix: restore @ts-nocheck on generated files stripped by buf generate
* fix: add circuit breaker + bootstrap to CII risk scores
Same pattern as theater posture (#948): replace fragile in-memory cache
+ manual persistent-cache with circuit breaker (SWR, IndexedDB, cooldown)
and bootstrap hydration. Eliminates learning-mode delay on cold start
and survives RPC failures without blanking the panel.
* fix: add localStorage sync prime for CII risk scores
getCachedScores() is called synchronously by country-intel.ts as a
fallback during learning mode. Without localStorage priming, the
breaker's async IndexedDB hydration hasn't run yet and returns null.
- Add shape validator (isValidCiiEntry) for untrusted localStorage data
- Add loadFromStorage/saveToStorage with 24h staleness ceiling
- Prime breaker synchronously at module load from localStorage
- Skip priming for empty cii arrays to avoid cached-empty trap
- Save to localStorage on both bootstrap and RPC success paths
Add circuit breaker + IndexedDB persistence + bootstrap hydration to
theater posture fetching — the only major panel without these resilience
layers. Replaces fragile in-memory cache (15-min TTL) and destructive
localStorage (30-min hard-delete) with the standard 3-tier pattern used
by all other panels.
* enhance supply chain panel
* fix(supply-chain): resolve P1 threat zeroing and P2 geo-first misclassification
P1: threat baseline is now always applied regardless of config
staleness — stale config only adds a review-recommended note,
never zeros the score.
P2: resolveChokepointId now checks text evidence first and only
falls back to proximity when text has no confident match.
Adds regression test: text "Bab el-Mandeb" with location near
Suez correctly resolves to bab_el_mandeb.
---------
Co-authored-by: fayez bast <fayezbast15@gmail.com>
GDELT GEO API had 99.9% timeout rate on Vercel Edge (746 invocations, ~31s
sequential calls vs 25s edge limit). Move fetching to Railway cron (15min),
write to Redis, have Vercel serve read-only from cache with bootstrap hydration.
- Add startPositiveEventsSeedLoop() to ais-relay.cjs (3 queries, dedup, classify)
- Rewrite handler to cache-read-only pattern (matches UCDP)
- Register bootstrap key in FAST_KEYS for instant first render
- Wire getHydratedData() in data-loader before RPC fallback
* feat(tech-readiness): bootstrap hydration via Railway seed + bootstrap key
Add pre-computed TechReadiness rankings to the bootstrap payload so the
panel renders immediately on first load instead of waiting for 4 slow
World Bank RPC calls (which can trip circuit breakers on cold starts,
causing persistent "No data available" until the 5-min cooldown expires).
- scripts/seed-wb-indicators.mjs: new Railway seed script that fetches
IT.NET.USER.ZS / IT.CEL.SETS.P2 / IT.NET.BBND.P2 / GB.XPD.RSDV.GD.ZS
for all countries, computes rankings (same weights as the frontend
getTechReadinessRankings), and writes economic:worldbank-techreadiness:v1
to Redis with a 7-day TTL
- api/bootstrap.js: register techReadiness key in BOOTSTRAP_CACHE_KEYS
and SLOW_KEYS (s-maxage=3600, appropriate for annual WB data)
- src/services/economic/index.ts: fast-path in getTechReadinessRankings()
returns getHydratedData('techReadiness') immediately on first page load;
country-specific comparison requests still use live RPCs
* ci: add weekly GHA workflow for WB tech readiness seed
Markets and commodities panels showed "Failed to load" because they
relied entirely on the listMarketQuotes RPC while sectors worked via
bootstrap hydration. Both also shared a single circuit breaker — 2
transient failures across both calls triggered a 5-minute cooldown.
- Add bootstrap Redis keys (market:stocks-bootstrap:v1 and
market:commodities-bootstrap:v1) to Railway seed and bootstrap API
- Hydrate markets/commodities from bootstrap on page load (same
pattern as sectors)
- Split circuit breaker: separate stockBreaker and commodityBreaker
so commodity failures don't kill market retries and vice versa
Split bootstrap endpoint into slow-changing (1h TTL: BIS rates,
minerals, sectors, etc.) and fast-changing (10min TTL: earthquakes,
outages, macro signals, etc.) tiers via ?tier=slow|fast query param.
Frontend fetches both tiers in parallel with shared 800ms timeout.
Partial failure is graceful — panels fall through to individual RPCs.
Backward compatible: no ?tier= param returns all keys at s-maxage=600.
Also removes orphaned ucdpEvents key (no getHydratedData consumer).
* feat(conflict): wire UCDP API access token across full stack
UCDP API now requires an `x-ucdp-access-token` header. Renames the
stub `UC_DP_KEY` to `UCDP_ACCESS_TOKEN` (matching ACLED convention)
and wires it through Rust keychain, sidecar allowlist + verification,
handler fetch headers, feature toggles, and desktop settings UI.
- Rename UC_DP_KEY → UCDP_ACCESS_TOKEN in type system and labels
- Add ucdpConflicts feature toggle with required secret
- Add UCDP_ACCESS_TOKEN to Rust SUPPORTED_SECRET_KEYS (24→25)
- Add sidecar ALLOWED_ENV_KEYS entry + validation with dynamic GED version probing
- Handler sends x-ucdp-access-token header when token is present
- UC_DP_KEY fallback in handler for one-release migration window
- Update .env.example, desktop-readiness, and docs
* feat(conflict): pre-fetch UCDP events via Railway cron + Redis cache
Replace the 228-line edge handler that fetched UCDP GED API on every
request with a thin Redis reader. The heavy fetch logic (version
discovery, paginated backward fetch, 1-year trailing window filter)
now runs as a setInterval loop in the Railway relay (ais-relay.cjs)
every 6 hours, writing to Redis key conflict:ucdp-events:v1.
Changes:
- Add UCDP seed loop to ais-relay.cjs (6h interval, 6 pages, 2K cap)
- Rewrite list-ucdp-events.ts as thin Redis reader (35 lines)
- Add conflict:ucdp-events:v1 to bootstrap batch keys
- Protect key from cache-purge via durable data prefix
- Add manual-only seed-ucdp-events workflow + standalone script
- Rename panel "UCDP Events" → "Armed Conflict Events" in locale
- Add 24h TTL + 25h staleness check as safety nets
* perf: bootstrap endpoint + polling optimization (phases 3-4)
Replace 15+ individual RPC calls on startup with a single /api/bootstrap
batch call that fetches pre-cached data from Redis. Consolidate 6 panel
setInterval timers into the central RefreshScheduler for hidden-tab
awareness (10x multiplier) and adaptive backoff (up to 4x for unchanged
data). Convert IntelligenceGapBadge from 10s polling to event-driven
updates with 60s safety fallback.
* fix(bootstrap): inline Redis + cache keys in edge function
Vercel Edge Functions cannot resolve cross-directory TypeScript imports
from server/_shared/. Inline getCachedJsonBatch and BOOTSTRAP_CACHE_KEYS
directly in api/bootstrap.js. Add sync test to ensure inlined keys stay
in sync with the canonical server/_shared/cache-keys.ts registry.
* test: add Edge Function module isolation guard for all api/*.js files
Prevents any Edge Function from importing from ../server/ or ../src/
which breaks Vercel builds. Scans all 12 non-helper Edge Functions.
* fix(bootstrap): read unprefixed cache keys on all environments
Preview deploys set VERCEL_ENV=preview which caused getKeyPrefix() to
prefix Redis keys with preview:<sha>:, but handlers only write to
unprefixed keys on production. Bootstrap is a read-only consumer of
production cache — always read unprefixed keys.
* fix(bootstrap): wire sectors hydration + add coverage guard
- Wire getHydratedData('sectors') in data-loader to skip Yahoo Finance
fetch when bootstrap provides sector data
- Add test ensuring every bootstrap key has a getHydratedData consumer
— prevents adding keys without wiring them
* fix(server): resolve 25 TypeScript errors + add server typecheck to CI
- _shared.ts: remove unused `delay` variable
- list-etf-flows.ts: add missing `rateLimited` field to 3 return literals
- list-market-quotes.ts: add missing `rateLimited` field to 4 return literals
- get-cable-health.ts: add non-null assertions for regex groups and array access
- list-positive-geo-events.ts: add non-null assertion for array index
- get-chokepoint-status.ts: add required fields to request objects
- CI: run `typecheck:api` (tsconfig.api.json) alongside `typecheck` to catch
server/ TS errors before merge
