* fix(tech-events): prevent partial fetch results from being cached
Techmeme ICS and dev.events RSS fetches on Vercel edge can partially
fail (timeout, truncation), returning only 1 event instead of 20+.
The handler cached this partial result for 6 hours, causing the Tech
Events panel to show empty.
- Add 8s AbortSignal.timeout on both external fetches
- Require minimum 5 events before caching (at least curated count)
* fix(tech-events): remove MIN_EVENTS threshold and add diagnostic logging
The MIN_EVENTS=5 threshold caused empty results when both external
sources fail on Vercel edge (only 4 curated events available). Now
any events > 0 are cached. Added detailed logging to diagnose why
Techmeme ICS and dev.events RSS fetches fail on Vercel edge.
Also removed past STEP Dubai 2026 event.
* fix(tech-events): route fetches through Railway relay when direct fails
Vercel edge functions cannot reliably reach Techmeme ICS and dev.events
RSS (datacenter IP blocking). Added fetchTextWithRelay() that tries
direct fetch first, then falls back to Railway relay proxy (/rss endpoint)
which fetches from a different IP. Same pattern used by news feed digest
and other handlers that hit blocked external sources.
* feat(tech-events): gold standard pipeline with Railway seed + bootstrap hydration
Full data pipeline overhaul to match project conventions:
- Add tech events seed loop to ais-relay.cjs: fetches Techmeme ICS +
dev.events RSS every 6h from Railway (avoids Vercel IP blocking),
parses both sources, merges with curated fallback events, writes to
Redis (data key + bootstrap key + seed-meta)
- Register in api/bootstrap.js BOOTSTRAP_CACHE_KEYS (SLOW tier)
- Register in api/health.js BOOTSTRAP_KEYS + SEED_META (420min stale)
- Restructure RPC handler: reads from single broad Redis key (populated
by seed), applies geocoding + filtering in-memory per request params.
Fallback fetcher only runs on cold start before first seed
- TechEventsPanel: check getHydratedData('techEvents') from bootstrap
before falling back to RPC call
- data-loader: use hydrated bootstrap data for map layer, RPC fallback
* feat(pro): harden enterprise contact form with mandatory fields and lead qualification
- Add mandatory phone number and company fields (client + server validation)
- Block free email domains (gmail, yahoo, hotmail, etc.) with 422 response and inline error
- Include phone (clickable tel: link) and email domain (clickable company link) in sales notification
- Add i18n translations for phone placeholder and work email error across all 21 locales
- Tighten phone regex to require start/end with digit, rejecting junk input
* fix(pro): rebuild static assets and fix contact handler tests
- Rebuild public/pro/ bundle to include new phone/company/email validation fields
- Add phone field to test validBody() fixture
- Add tests for free email domain rejection (422), missing org, missing/invalid phone
- Add AVIATIONSTACK and NOTAM proto enum values for accurate source attribution
- AviationStack flight data alerts now show "Flight Data" instead of "Computed"
- NOTAM closure/restriction alerts now show "NOTAM"
- Remove generateSimulatedDelay() fallback that produced fake random alerts
- Reduce all aviation cache TTLs from 2h to 30min for fresher data
- Reduce relay seed interval from 1h to 30min, TTL from 4h to 1h
- Reduce seed freshness threshold from 45min to 20min
- Update health check maxStaleMin from 90 to 60min
- Update all 21 locale files with new source labels
* fix(health): fix riskScores seeding gap and seed-meta key mismatch
- Switch RPC handler to cachedFetchJsonWithMeta so stale key is refreshed
on every successful response (cache hit or miss), not just cache misses
- Fix seed-meta key mismatch: health.js and seed-health.js now check
seed-meta:risk:scores:sebuf (matching what cachedFetchJson writes)
- Add warm-ping loop in relay (8min interval) to keep RPC cache fresh
- Remove dead startCiiSeedLoop and 345 lines of unused CII seed code
* fix(scoring): await stale key write to prevent edge runtime drop
Edge/serverless runtimes may terminate the isolate before a
fire-and-forget Redis write completes. Await the setCachedJson
call so the stale key TTL is guaranteed to be extended.
* fix(contact): add dedicated enterprise contact endpoint with storage and email notifications
The enterprise contact form was posting to /api/register-interest which
only stored email for the waitlist. Name, organization, and message fields
were silently dropped and no notification was sent.
- Add api/contact.js endpoint with Turnstile, rate limiting, honeypot
- Add Convex contactMessages table and submit mutation
- Send notification email to sales@worldmonitor.app via Resend
- Sanitize email subject (strip newlines, truncate length)
- Fix cf-connecting-ip priority in register-interest.js IP detection
* chore: add contact.js to legacy endpoint allowlist
* fix(contact): harden Turnstile enforcement, surface email status, add tests
- Turnstile rejects in production when TURNSTILE_SECRET_KEY is unset
(only allows skip in development)
- sendNotificationEmail returns boolean, response includes emailSent field
- Log error (not warn) when RESEND_API_KEY is missing in production
- Add 15 endpoint tests covering validation, Turnstile, notifications, Convex
* feat(map): add NOTAM overlay + satellite imagery integration
NOTAM Overlay:
- Expand airport monitoring from MENA-only to 64 global airports
- Add ScatterplotLayer (55km red rings) on flat map for airspace closures
- Add CSS-pulsing ring markers on globe for closures
- Independent of flights layer toggle (works when flights OFF)
- Bump NOTAM cache key v1 to v2
Satellite Imagery:
- Add Capella SAR STAC catalog proxy at /api/imagery/v1
- SSRF protection via URL allowlist + bbox/datetime validation
- SatelliteImageryPanel with preview thumbnails and scene metadata
- PolygonLayer footprints on flat map with viewport-triggered search
- Polygon footprints on globe with "Search this area" button
- Full variant only, default disabled
Layer key propagation across all 23+ files including variants,
harnesses, registry, URL state, and renderer channels.
* fix(imagery): wire panel data flow, fix viewport race, add datetime filter
P1 fixes:
- Imagery scenes now flow through MapContainer.setOnImageryUpdate()
callback, making data available to both renderers and panel
- Add version guard to fetchImageryForViewport() preventing stale
responses from overwriting newer viewport data
- Wire SatelliteImageryPanel.update() and setOnSearchArea() in
panel-layout.ts (panel was previously unhooked)
- Globe mode "Search this area" fetches via MapContainer.getBbox()
P2 fix:
- search-imagery.ts now filters STAC items by datetime range when
the client provides the datetime parameter
Also:
- Add MapContainer.getBbox() for viewport-aware imagery fetching
- Add DeckGLMap.getBbox() public method
- Data-loader layer toggle triggers initial imagery fetch
* fix(imagery): complete source filter + fix date-only end bound
- Filter STAC items by constellation when source param is provided,
making the API contract match actual behavior
- Date-only end bounds (YYYY-MM-DD without T) now include the full
day (23:59:59.999Z) instead of only midnight
* fix: resolve YouTube 'sign in to confirm' bot-check in embed panels
YouTube was showing a bot-verification prompt in the LiveWebcamsPanel
and LiveNewsPanel despite the user being logged into YouTube in the
same browser session.
LiveWebcamsPanel (primary fix):
- Changed embed domain from youtube-nocookie.com to youtube.com.
The nocookie domain deliberately strips all cookies, so YouTube
can never verify a signed-in session.
- Removed sandbox attribute which blocked the Storage Access API
(allow-storage-access-by-user-activation was missing).
- Added storage-access to iframe allow attribute.
LiveNewsPanel:
- renderDesktopEmbed now passes origin and parentOrigin query params
so postMessage is not silently dropped by the embed.
- Added storage-access to iframe allow attribute.
- Fixed MutationObserver target: was watching this.playerElement but
YT.Player(domElement) replaces that div in its parent, so the
observer never fired. Now observes playerContainer with a YouTube
iframe filter, and YT.Player receives the element ID string so the
iframe is inserted as a child of the div instead.
local-api-server.mjs (youtube-embed handler):
- MutationObserver patches inner YouTube iframe with storage-access.
- Added Permissions-Policy: storage-access=* response header.
- Embed page calls document.requestStorageAccess() on load.
api/youtube/embed.js (Vercel/edge path):
- Added tauri://localhost to ALLOWED_PARENT_ORIGINS.
- Added Permissions-Policy: storage-access=* response header.
- Embed page calls document.requestStorageAccess() on load.
* fix(pr-review): address review feedback on YouTube Storage Access API changes
- LiveWebcamsPanel: tested allow-storage-access-by-user-activation sandbox token
as suggested; reverted — Chrome silently blocks Storage Access API even with
the token present. Documented why sandbox removal is the only working approach.
- LiveWebcamsPanel: added comment documenting youtube-nocookie→youtube.com
privacy trade-off as intentional.
- LiveNewsPanel: wrap YT.Player constructor in try/catch to disconnect
storageObserver on error; add 10 s auto-disconnect timeout to prevent leaks.
- embed.js + local-api-server.mjs: scope permissions-policy storage-access to
self + youtube.com rather than *.
- embed.js + local-api-server.mjs: add gesture-gated requestStorageAccess()
fallback on first user interaction.
- embed.js: remove duplicate tauri://localhost from ALLOWED_PARENT_ORIGINS
(already covered via ALLOWED_ORIGINS spread).
* fix(review): gate sidecar patch on storage-access, revert web webcam path
1. Sidecar MutationObserver: gate iframe patch on storage-access absence
instead of autoplay absence. If YouTube ships iframes with autoplay
already present, the old check would skip adding storage-access entirely.
2. Web webcam path: revert to youtube-nocookie.com and restore sandbox.
The raw YouTube iframe cannot call requestStorageAccess() (no controlled
bridge document), so switching to youtube.com only regressed privacy
and sandbox security without actually fixing the bot-check.
---------
Co-authored-by: Elie Habib <elie.habib@gmail.com>
* fix(health): separate status severity, expand seed-health domains, harden Redis errors
- Distinguish WARNING (warns only, 200) from DEGRADED (few crits, 503)
- Exclude OK_CASCADE entries from compact mode problems list
- Add missing dataSize properties (sectors, statuses, scores)
- Remove redisKey from public /api/health responses (info disclosure)
- Expand seed-health.js from 12 to 40 domains aligned with health.js SEED_META
- Return 503 for stale seeds, 200 for missing (cold start) in seed-health
- Throw on Redis config/HTTP errors instead of masking as empty results
* fix(seed-health): align severity with health.js, remove RPC-only domains
- Revert severity order: missing = degraded/503, stale = warning/200
(matches health.js where empty/missing is higher severity than stale)
- Remove RPC-populated domains (BIS, minerals, giving, worldbank,
macro-signals) whose seed-meta is only written on-demand by
cachedFetchJson, not by scheduled seeders
* fix(seed-health): keep 200 for all states to avoid breaking migration validator
scripts/validate-seed-migration.mjs (L203) treats non-200 as hard failure
and skips body parsing. Returning 503 for degraded would break that flow.
Keep 200 and let callers interpret the overall field instead.
Move theaterPosture from SLOW (2h CDN) to FAST tier (20min/10min after
PR #1314) so military posture data stays fresh. Increase risk scores
breaker TTL to 30min to match health.js maxStaleMin, and reduce
localStorage staleness from 24h to 1h to prevent stale risk data in UI.
CDN-Cache-Control for fast tier was 20min + 5min stale-while-revalidate,
but insights-loader rejects data older than 15min. This caused frequent
fallback to the slow 4-step client-side AI pipeline. Align CDN TTL with
origin Cache-Control (10min + 2min SWR) so CF revalidates before the
freshness threshold.
* Add premium finance stock analysis suite
* docs: link premium finance from README
Add Premium Stock Analysis entry to the Finance & Markets section
with a link to docs/PREMIUM_FINANCE.md.
* fix: address review feedback on premium finance suite
- Chunk Redis pipelines into batches of 200 (Upstash limit)
- Add try-catch around cachedFetchJson in backtest handler
- Log warnings on Redis pipeline HTTP failures
- Include name in analyze-stock cache key to avoid collisions
- Change analyze-stock and backtest-stock gateway cache to 'slow'
- Add dedup guard for concurrent ledger generation
- Add SerpAPI date pre-filter (tbs=qdr:d/w)
- Extract sanitizeSymbol to shared module
- Extract buildEmptyAnalysisResponse helper
- Fix RSI to use Wilder's smoothing (matches TradingView)
- Add console.warn for daily brief summarization errors
- Fall back to stale data in loadStockBacktest on error
- Make daily-market-brief premium on all platforms
- Use word boundaries for short token headline matching
- Add stock-analysis 15-min refresh interval
- Stagger stock-analysis and backtest requests (200ms)
- Rename signalTone to stockSignalTone
Track ~80-120 intelligence-relevant satellites on the 3D globe using CelesTrak
TLE data and client-side SGP4 propagation (satellite.js). Satellites render at
actual orbital altitude with country-coded colors, 15-min orbit trails, and
ground footprint projections.
Architecture: Railway seeds TLEs every 2h → Redis → Vercel CDN (1h cache) →
browser does SGP4 math every 3s (zero server cost for real-time movement).
- New relay seed loop (ais-relay.cjs) fetching military + resource groups
- New edge handler (api/satellites.js) with 10min cache + negative cache
- Frontend service with circuit breaker and propagation lifecycle
- GlobeMap integration: markers, trails (pathsData), footprints, tooltips
- Layer registry as globe-only "Orbital Surveillance" with i18n (21 locales)
- Full documentation at docs/ORBITAL_SURVEILLANCE.md with roadmap
- Fix pre-existing SearchModal TS error (non-null assertion)
* fix(relay): compute theater posture directly instead of pinging Vercel RPC
The relay's theater posture seed loop was pinging the Vercel RPC endpoint
which itself needed WS_RELAY_URL to proxy OpenSky back through the relay —
a circular dependency that resulted in empty theaters.
Now the relay fetches OpenSky directly via its own localhost proxy,
applies military callsign filtering, computes postures for all 9 theaters,
and writes all 3 Redis keys (live/stale/backup) + seed-meta directly.
Wingbits API serves as fallback when OpenSky is unavailable.
* feat(relay): seed military flights to Redis, rewire frontend to read from edge handler
Relay fetches OpenSky every 5 min, filters by ICAO hex ranges (29) and
callsign patterns (83), converts units (m→ft, m/s→kts), and writes to
Redis. Frontend reads pre-filtered data via /api/military-flights edge
handler instead of proxying through OpenSky directly — eliminates
cascading 429 rate limits.
- Add seedMilitaryFlights() with in-flight mutex, 2-region fetch, empty guard
- Add api/military-flights.js edge handler (Redis → stale fallback)
- Rewrite src/services/military-flights.ts to fetchFromRedis()
- Add health.js tracking (STANDALONE_KEYS, SEED_META, CASCADE_GROUPS)
- Theater posture consumes shared flight data (no duplicate OpenSky calls)
* fix(relay): theater posture starvation, aircraftType downgrade, and openskyRelay gate
- Move lastMilFlightsSeedMs after successful posture passthrough so the
standalone fallback runs when passthrough keeps erroring
- Use f.aircraftType from military seed instead of re-classifying via
theaterDetectAircraftType (which has no fighter branch)
- Add militaryFlights feature flag with no required secrets; swap gate
from openskyRelay so desktop users get Redis-backed data without
legacy OpenSky credentials
* fix(relay): add fighter branch to theaterDetectAircraftType
The standalone theater posture fallback path could not classify fighter
callsigns (BOLT, VIPER, RAPTOR, etc.), causing them to be counted as
unknown and underreporting strike_capable posture levels.
* fix(military-flights): preserve OpenSky direct path for desktop, Redis for web
Desktop (Tauri) cannot reach /api/military-flights (Vercel edge handler).
Restore the original OpenSky direct fetch path for desktop mode while
using the new Redis-backed path for web/cloud. Route based on
isDesktopRuntime() — desktop uses openskyRelay feature gate + direct
OpenSky, web uses militaryFlights feature gate + Redis edge handler.
* feat(gpsjam): migrate GPS jamming from gpsjam.org to Wingbits API
Replace gpsjam.org CSV scraping with Wingbits customer API for GPS/GNSS
interference data. This is a proper API with structured JSON responses
instead of fragile web scraping.
Key changes:
- Rewrite fetch-gpsjam.mjs seeder for Wingbits API (x-api-key auth)
- Delete ~150-line gpsjam seed loop from ais-relay.cjs (now standalone cron)
- Simplify api/gpsjam.js to Redis-only reads with v1→v2 fallback
- Update data shape: pct/good/bad/total → npAvg/sampleCount/aircraftCount
- Redis key: intelligence:gpsjam:v1 → v2 (with dual-write transition)
- Add vite dev plugin for local development
- Update all frontend components (MapPopup, DeckGLMap, GlobeMap, locales)
Zero-downtime: seeder dual-writes both v1 and v2 keys, edge handler
falls back to v1 with shape normalization. Remove v1 code after 24-72h.
* fix(gpsjam): improve v1 fallback normalization and update all locale files
- v1 fallback now derives npAvg from severity thresholds (high: 0.3,
medium: 0.8) instead of hardcoding 0, uses bad/total for counts
- Update all 20 non-English locale files to use new gpsJamming keys
(navPerformance, samples, aircraft) with English fallback values
* fix(gpsjam): dual-write v1 in old schema shape and catch Redis errors
- Seeder now converts v2 data back to v1 shape (pct/good/bad/total)
for the dual-write, so old deployments and rollbacks parse correctly
- Edge handler wraps readFromRedis calls in try-catch so network/timeout
errors return graceful 503 instead of platform 500s
- Add World Bank seed loop to ais-relay (24h interval) for techReadiness,
progressData, and renewableEnergy — previously manual-only with no cron
- Add seed-meta writes for UCDP and GPSJAM relay loops so health endpoint
can track freshness (was always showing STALE/unknown)
- Fix theater posture and service statuses RPC URLs from worldmonitor.app
to api.worldmonitor.app to bypass Cloudflare bot protection (403)
- Adjust UCDP maxStaleMin from 60 to 420 to match 6h relay interval
- Remove position badge, show "You're in!" for new registrations
- Convex: auto-generate referral code for old records missing one
- Add email send logging (RESEND_API_KEY missing/error/success)
- Add youreIn translations to all 21 locales
* fix: eliminate frontend external API calls, enforce gold standard pattern
- Polymarket: remove browser fan-out (536→105 lines), bootstrap → RPC only
- USASpending: remove direct API calls, read from bootstrap hydration
- NWS Weather: remove direct API calls, read from bootstrap hydration
- Nominatim: proxy through api/reverse-geocode.js with Redis cache + SSRF clamping
- Add seed scripts for weather alerts (15min) and spending (60min)
- Wire both seed loops into ais-relay.cjs
- Register weatherAlerts + spending in bootstrap.js and health.js
- Add 4 missing standalone keys to health.js (cyberThreatsRpc, militaryBases, temporalAnomalies, displacement)
* fix: resolve reload regressions and null-cache poisoning from #1217
- Weather/Spending: fall back to `/api/bootstrap?keys=` on scheduled
reloads after the one-shot `getHydratedData()` is consumed
- Prediction: add client-side bootstrap filter for country markets
when RPC fails (server skips bootstrap for query-based requests)
- Reverse-geocode: restore abort/timeout guard so transient network
errors don't permanently poison the in-memory cache
* perf(baseline): move temporal baseline for news+fires to server-side
Every browser client was calling record-baseline-snapshot (POST) +
get-temporal-baseline (GET) on every data refresh from 5 call sites.
With N concurrent users this created N identical writes and ~5N reads
per cycle — causing 429 rate limiting and statistically biased baselines.
Phase 1 moves news and satellite_fires to server-side computation:
- New ListTemporalAnomalies RPC reads counts from existing Redis keys
(news:insights:v1, wildfire:fires:v1), computes anomalies against v2
baselines, applies Welford update (1 sample/cycle), caches 15min
- Bootstrap hydration delivers anomalies on page load (zero extra calls)
- Client refreshes via RPC every 10min (1 cached call vs 5N before)
- Remaining 3 types (military_flights, vessels, ais_gaps) stay client-side
- Owner-guarded distributed lock prevents concurrent computation
- All reads/writes use prefix-aware getCachedJson/setCachedJson
Expected ~60% reduction in baseline-related Vercel invocations.
* fix(temporal): per-invocation lock owner and immediate refresh on cold cache
P1: When bootstrap has no temporal anomaly data (cold cache, expired
snapshot, fresh deploy), fire refreshTemporalBaseline() immediately
instead of waiting up to 10 minutes for the scheduled refresh.
P2: Generate lockOwner per invocation via makeLockOwner() instead of
once at module load. Prevents warm edge isolates from cross-releasing
each other's locks when one invocation outlives the 30s TTL.
* fix(temporal): use TTL-only lock instead of TOCTOU GET-then-DEL release
The non-atomic GET→check-owner→DEL release was vulnerable to a race
where the TTL expires between GET and DEL, allowing a new lock holder
to be evicted. Simplify by relying solely on the 30s TTL for lock
expiry — the computation completes well within that window.
* perf(edge): reduce unnecessary Vercel edge invocations (#6 findings)
Phase 1 — Client-only fixes:
- Remove predictions double getHydratedData read from data-loader.ts;
fetchPredictions() handles hydration internally
- Fix UCDP delete-on-read race: read hydratedUcdp once in data-loader,
pass to both fetchUcdpClassifications() and fetchUcdpEvents()
Phase 2 — Batch RPCs (proto + server + client):
- Add GetHumanitarianSummaryBatch RPC: replaces 20-request HAPI fanout
with single batch call (getCachedJsonBatch + per-key Redis caching)
- Add GetFredSeriesBatch RPC: replaces 7-request FRED fanout with
single batch call (same pattern)
- Both batch RPCs have 404 deploy-skew fallback to per-item calls
Phase 3 — Seed gap:
- Add seed-service-statuses.mjs standalone seed script
- Add 15-min warm-ping loop in AIS relay for service statuses
- Remove serviceStatuses from ON_DEMAND_KEYS in health.js
Net savings: up to 28 edge calls eliminated on cold miss per page load.
* fix(edge): address code review findings (P1–P3)
P1: Fix dead 404 deploy-skew fallback — circuit breaker was swallowing
the ApiError before the catch block could detect it. Move 404 fallback
inside the breaker callback so it executes before the breaker catches.
P2: Replace 172-line seed-service-statuses.mjs (duplicated parser logic)
with a 60-line warm-ping that triggers the existing RPC handler.
P2: Extract shared ISO2_TO_ISO3 mapping to conflict/v1/_shared.ts,
eliminating duplication between single and batch HAPI handlers.
P3: Remove unnecessary UPSTASH_ENABLED guard from relay warm-ping
(it calls Vercel RPC, not Redis directly).
P3: Clean up unused per-series FRED breakers and fetchSingleFredSeries
(replaced by batch breaker). Update getFredStatus() accordingly.
* fix(edge): use concurrent fetches in batch handlers
HAPI batch: replace serial loop with groups of 5 concurrent fetches
using Promise.allSettled for partial-success resilience. Bump client
timeout to 60s (4 rounds × 15s upstream timeout worst case).
FRED batch: replace serial loop with fully parallel Promise.allSettled
(max 10 series, each hits separate FRED endpoint).
Both changes prevent empty-result regression on cold cache that the
serial approach caused when upstream latency exceeded the client timeout.
* fix(health): add seed-meta tracking for all bootstrap keys missing freshness data
6 bootstrap keys had no SEED_META entries in health.js, so health
endpoint could never track their freshness (always seedStale: true).
health.js:
- Add progressData + renewableEnergy to BOOTSTRAP_KEYS (missed in PR #1159)
- Add SEED_META entries for: positiveGeoEvents, riskScores, iranEvents,
ucdpEvents, sectors, techReadiness, progressData, renewableEnergy
ais-relay.cjs:
- Add seed-meta writes for positive events (GDELT), risk scores (CII),
and sectors — these loops had no freshness tracking
- iranEvents and ucdpEvents already write seed-meta via their seed scripts
* fix(seed): add seed-meta writes to seed-wb-indicators.mjs
The seed script wrote data keys but never wrote seed-meta keys,
causing health endpoint to report STALE_SEED for techReadiness,
progressData, and renewableEnergy indefinitely.
* fix(economic): seed all WB indicators on Railway, never call WB API from frontend
Extends seed-wb-indicators.mjs to pre-compute progress data (4 indicators)
and renewable energy data (EG.ELC.RNEW.ZS) alongside tech readiness rankings.
Frontend callers (progress-data.ts, renewable-energy-data.ts, getTechReadinessRankings,
getCountryComparison) now read exclusively from bootstrap/Redis seed data.
Zero Vercel Edge → World Bank API calls remain.
* fix: address code review findings (P1+P2)
- Fix triple JSON.parse in seed verification (P1)
- Graceful fallback for renewable data fetch failure (P2)
- Use Map lookup instead of Array.find in progress-data (P2)
- Update regression test for bootstrap-only getTechReadinessRankings (P2)
* feat(seeds): add BIS data seed job and relax health thresholds
Add seed-bis-data.mjs that fetches all 3 BIS datasets (policy rates,
exchange rates, credit-to-GDP) in parallel and writes to Redis. This
keeps the cache warm instead of relying on on-demand RPC calls.
Relax BIS health thresholds from 1440min (24h) to 2880min (48h) since
BIS data is monthly/quarterly — 24h was too aggressive.
* fix(health): relax minerals and giving thresholds to 7 days
Both are static/hardcoded data with no external API calls.
2880min (48h) was too aggressive for annual data.
* fix(gpsjam): write seed-meta for health freshness tracking
The fetch-gpsjam script seeded Redis data but never wrote
seed-meta:intelligence:gpsjam, causing health to report STALE_SEED.
Keys with data but no freshness proof should show STALE_SEED (WARN),
not silently pass as OK. Removes the grace period so health accurately
reflects whether data is both present AND fresh.
cachedFetchJson now auto-writes seed-meta:{key} on every fresh fetch,
enabling freshness tracking for all standalone RPC-populated keys
without manual instrumentation. Health endpoint checks staleness for
all monitored keys using configurable maxStaleMin thresholds.
Checks all 44 Redis cache keys (33 bootstrap + 11 standalone) plus
17 seed-meta freshness timestamps in a single Redis pipeline.
- Returns HEALTHY/DEGRADED/UNHEALTHY with per-key status
- Distinguishes seed-backed keys (STALE_SEED) from on-demand keys (EMPTY_ON_DEMAND)
- No auth required, ?compact=1 for minimal payload
- UptimeRobot: keyword monitor on "HEALTHY", HTTP 503 on UNHEALTHY
- Add 7 RSS feed categories for happy variant: positive, science, nature,
health, inspiring, community with sources like Mongabay, Yes! Magazine,
Shareable, Conservation Optimism, My Modern Met, GNN subcategories
- Switch GPS jamming layer from ScatterplotLayer (circles) to
H3HexagonLayer (tessellated hexagons) matching industry standard
- Fix layer toggle not visually updating by adding triggerRepaint()
after all deck.gl setProps() calls — MapboxOverlay in interleaved
mode requires MapLibre to repaint for changes to appear
Register 6 new seeds in bootstrap.js (crypto, gulf, stablecoin, unrest,
iran, ucdp) and wire getHydratedData() in 7 service files. Also adds
hydration for 2 previously-registered keys (cyberThreats, flightDelays)
that had no frontend consumer. Syncs cache-keys.ts with bootstrap.js
for test parity.
Cyber hydration correctly maps through toCyberThreat() to convert proto
enum strings to friendly types.
* perf: reduce Vercel data transfer costs with CDN optimization
- Increase polling intervals (markets 8→12min, feeds 15→20min, crypto 8→12min)
- Increase background tab hiddenMultiplier from 10→30 (polls 3x less when hidden)
- Double CDN s-maxage TTLs across all cache tiers in gateway
- Add CDN-Cache-Control header for Cloudflare-specific longer edge caching
- Add ETag generation + 304 Not Modified support in gateway (zero-byte revalidation)
- Add CDN-Cache-Control to bootstrap endpoint
- Add explicit SPA rewrite rule in vercel.json for CF proxy compatibility
- Add Cache-Control headers for /map-styles/, /data/, /textures/ static paths
* fix: prevent CF from caching SPA HTML + reduce Polymarket bandwidth 95%
- vercel.json: apply no-cache headers to ALL SPA routes (same regex as
rewrite rule), not just / and /index.html — prevents CF proxy from
caching stale HTML that references old content-hashed bundle filenames
- Polymarket: add server-side aggregation via Railway seed script that
fetches all tags once and writes to Redis, eliminating 11-request
fan-out per user per poll cycle
- Bootstrap: add predictions to hydration keys for zero-cost page load
- RPC handler: read Railway-seeded bootstrap key before falling back to
live Gamma API fetch
- Client: 3-strategy waterfall (bootstrap → RPC → fan-out fallback)
- register-interest.js: coerce source/appVersion to string with a 100-char cap
before forwarding to Convex. Non-string values (objects, arrays) are truthy so
the previous || 'unknown' guard passed them through, causing Convex to throw
a type-validation error and surface a 500 to the caller. Also fixes unbounded
metadata strings filling the registrations table cheaply.
- rss-proxy.js: apply the same www-normalization used by the initial domain check
to the 301-redirect hostname check. The old bare ALLOWED_DOMAINS.includes(hostname)
call rejected canonical redirects (e.g. bbc.co.uk -> www.bbc.co.uk) even when
one form is allowlisted, breaking several feeds silently.
- _api-key.js: align BROWSER_ORIGIN_PATTERNS Vercel-preview regex with the
narrower pattern already enforced by _cors.js (worldmonitor-*-elie-*.vercel.app).
The broader worldmonitor-*.vercel.app pattern was dead code because _cors.js
rejects those origins before _api-key.js is reached.
When CDN serves stale bootstrap responses with empty arrays (e.g.,
{anomalies:[]}), the hydration check `if (hydrated)` is truthy, causing
panels to skip the RPC fallback and show empty data. Fixed all 12
vulnerable getHydratedData consumers to verify the payload has actual
data before trusting it.
Also rewrote api/telegram-feed.js to inspect response body — empty
responses get short CDN TTL (15s) to prevent prolonged staleness.
* feat: Implement comprehensive aviation monitoring service with flight search, status, news, and tracking.
* feat: Introduce Airline Intelligence Panel with aviation data tabs, map components, and localization.
* feat: Implement DeckGL-based map for advanced visualization, D3/SVG fallback, i18n support, and aircraft tracking.
* Update server/worldmonitor/aviation/v1/get-carrier-ops.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/search-flight-prices.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/track-aircraft.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/get-airport-ops-summary.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update proto/worldmonitor/aviation/v1/position_sample.proto
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/list-airport-flights.ts
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update proto/worldmonitor/aviation/v1/price_quote.proto
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: Add server-side endpoints for aviation news and aircraft tracking, and introduce a new DeckGLMap component for map visualization.
* Update server/worldmonitor/aviation/v1/list-airport-flights.ts
The cache key for listAirportFlights excludes limit, but the upstream fetch/simulated generator uses limit to determine how many flights to return. If the first request within TTL uses a small limit, larger subsequent requests will be incorrectly capped until cache expiry. Include limit (or a normalized bucket/max) in cacheKey, or always fetch/cache a fixed max then slice per request.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* Update server/worldmonitor/aviation/v1/get-flight-status.ts
getFlightStatus accepts origin, but cacheKey does not include it. This can serve cached results from an origin-less query to an origin-filtered query (or vice versa). Add origin (normalized) to the cache key or apply filtering after fetch to ensure cache correctness.
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: Implement DeckGL map for advanced visualization and new aviation data services.
* fix(aviation): prevent cache poisoning and keyboard shortcut in inputs
- get-carrier-ops: move minFlights filter post-cache to avoid cache
fragmentation (different callers sharing cached full result)
- AviationCommandBar: guard Ctrl+J shortcut so it does not fire when
focus is inside an INPUT or TEXTAREA element
Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>
* feat: introduce AviationCommandBar component for parsing user commands, fetching aviation data, and displaying results.
* feat: Implement aircraft tracking service with OpenSky and simulated data sources.
* feat: introduce DeckGLMap component for WebGL-accelerated map visualizations using deck.gl and maplibre-gl.
* fix(aviation): address code review findings for PR #907
Proto: add missing (sebuf.http.query) annotations on all GET request
fields across 6 proto files; add currency/market fields to
SearchFlightPricesRequest.
Server: add parseStringArray to aviation _shared.ts and apply to
get-airport-ops-summary, get-carrier-ops, list-aviation-news handlers
to prevent crash on comma-separated query params; remove leaked API
token from URL params in travelpayouts_data; fix identical simulated
flight statuses in list-airport-flights; remove unused endDate var;
normalize cache key entity casing in list-aviation-news.
Client: refactor AirlineIntelPanel to extend Panel base class and
register in DEFAULT_PANELS for full/tech/finance variants; fix
AviationCommandBar reference leak with proper destroy() cleanup in
panel-layout; rename priceUsd→priceAmount in display type and all
usages; change auto-refresh to call refresh() instead of loadOps().
Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
* feat: introduce aviation command bar component with aircraft tracking and flight information services.
* feat: Add `AirlineIntelPanel` component for displaying airline operations, flights, carriers, tracking, news, and prices in a tabbed interface.
* feat: Add endpoints for listing airport flights and fetching aviation news.
* Update proto/worldmonitor/aviation/v1/search_flight_prices.proto
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
* feat: Add server endpoint for listing airport flights and client-side MapPopup types and utilities.
* feat: Introduce MapPopup component with support for various data types and responsive positioning for map features.
* feat: Add initial English localization file (en.json).
* fix(aviation): address PR review findings across aviation stack
- Add User-Agent header to Travelpayouts provider (server convention)
- Use URLSearchParams for API keys instead of raw URL interpolation
- Add input length validation on flightNumber (max 10 chars)
- Replace regex XML parsing with fast-xml-parser in aviation news
- Fix (f as any)._airport type escape with typed Map<FI, string>
- Extract DEFAULT_WATCHED_AIRPORTS constant from hardcoded arrays
- Use event delegation for AirlineIntelPanel price search listener
- Add bootstrap hydration key for flight delays
- Bump OpenSky cache TTL to 120s (anonymous tier rate limit)
- Match DeckGLMap aircraft poll interval to server cache (120s)
- Fix GeoJSON polygon winding order (shoelace check + auto-reversal)
* docs: add aviation env vars to .env.example
AVIATIONSTACK_API, ICAO_API_KEY, TRAVELPAYOUTS_API_TOKEN
* feat: Add aviation news listing API and introduce shared RSS allowed domains.
* fix: add trailing newline to rss-allowed-domains.json, remove unused ringIsClockwise
---------
Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
* perf(rss): route RSS direct to Railway, skip Vercel middleman
Vercel /api/rss-proxy has 65% error rate (207K failed invocations/12h).
Route browser RSS requests directly to Railway (proxy.worldmonitor.app)
via Cloudflare CDN, eliminating Vercel as middleman.
- Add VITE_RSS_DIRECT_TO_RELAY feature flag (default off) for staged rollout
- Centralize RSS proxy URL in rssProxyUrl() with desktop/dev/prod routing
- Make Railway /rss public (skip auth, keep rate limiting with CF-Connecting-IP)
- Add wildcard *.worldmonitor.app CORS + always emit Vary: Origin on /rss
- Extract ~290 RSS domains to shared/rss-allowed-domains.cjs (single source of truth)
- Convert Railway domain check to Set for O(1) lookups
- Remove rss-proxy from KEYED_CLOUD_API_PATTERN (no longer needs API key header)
- Add edge function test for shared domain list import
* fix(edge): replace node:module with JSON import for edge-compatible RSS domains
api/_rss-allowed-domains.js used createRequire from node:module which is
unsupported in Vercel Edge Runtime, breaking all edge functions (including
api/gpsjam). Replaced with JSON import attribute syntax that works in both
esbuild (Vercel build) and Node.js 22+ (tests).
Also fixed middleware.ts TS18048 error where VARIANT_OG[variant] could be
undefined.
* test(edge): add guard against node: built-in imports in api/ files
Scans ALL api/*.js files (including _ helpers) for node: module imports
which are unsupported in Vercel Edge Runtime. This would have caught the
createRequire(node:module) bug before it reached Vercel.
* fix(edge): inline domain array and remove NextResponse reference
- Replace `import ... with { type: 'json' }` in _rss-allowed-domains.js
with inline array — Vercel esbuild doesn't support import attributes
- Replace `NextResponse.next()` with bare `return` in middleware.ts —
NextResponse was never imported
* ci(pre-push): add esbuild bundle check and edge function tests
The pre-push hook now catches Vercel build failures locally:
- esbuild bundles each api/*.js entrypoint (catches import attribute
syntax, missing modules, and other bundler errors)
- runs edge function test suite (node: imports, module isolation)
* fix: add circuit breaker + bootstrap to CII risk scores
Same pattern as theater posture (#948): replace fragile in-memory cache
+ manual persistent-cache with circuit breaker (SWR, IndexedDB, cooldown)
and bootstrap hydration. Eliminates learning-mode delay on cold start
and survives RPC failures without blanking the panel.
* fix: add localStorage sync prime for CII risk scores
getCachedScores() is called synchronously by country-intel.ts as a
fallback during learning mode. Without localStorage priming, the
breaker's async IndexedDB hydration hasn't run yet and returns null.
- Add shape validator (isValidCiiEntry) for untrusted localStorage data
- Add loadFromStorage/saveToStorage with 24h staleness ceiling
- Prime breaker synchronously at module load from localStorage
- Skip priming for empty cii arrays to avoid cached-empty trap
- Save to localStorage on both bootstrap and RPC success paths
* feat: Railway CII seed + bootstrap hydration for instant panel render
- Add 8-source CII seed to Railway (ACLED, UCDP, outages, climate, cyber, fires, GPS, Iran strikes)
- Neuter Vercel handler to read-only (returns Railway-seeded cache, never recomputes)
- Register riskScores in bootstrap FAST tier for CDN-cached delivery
- Add early CII hydration in data-loader before intelligence signals
- Add CIIPanel.renderFromCached() for instant render from bootstrap data
- Refactor cached-risk-scores.ts: circuit breaker + localStorage sync prime + bootstrap hydration
- Progressive enhancement: cached render → full 18-source local recompute (no spinner)
* fix: remove duplicate riskScores key in BOOTSTRAP_TIERS after merge
* feat: move EONET/GDACS to server-side with Redis caching and bootstrap hydration
Browser-direct fetches to eonet.gsfc.nasa.gov and gdacs.org caused CORS
errors and had no server-side caching. This moves both to the standard
Vercel edge → cachedFetchJson → Redis → bootstrap hydration pattern.
- Add proto definitions for NaturalService with ListNaturalEvents RPC
- Create server handler merging EONET + GDACS with 30min Redis TTL
- Add Vercel edge function at /api/natural/v1/list-natural-events
- Register naturalEvents in bootstrap SLOW_KEYS for CDN hydration
- Replace browser-direct fetches with RPC client + circuit breaker
- Delete src/services/gdacs.ts (logic moved server-side)
* fix: restore @ts-nocheck on generated files stripped by buf generate
The AU Smartraveller RSS feeds have been consistently returning 503
from both Vercel edge and Railway relay. Remove all references from
security-advisories feeds, rss-proxy allowed domains, and relay allowlist.
* fix: add circuit breaker + bootstrap to CII risk scores
Same pattern as theater posture (#948): replace fragile in-memory cache
+ manual persistent-cache with circuit breaker (SWR, IndexedDB, cooldown)
and bootstrap hydration. Eliminates learning-mode delay on cold start
and survives RPC failures without blanking the panel.
* fix: add localStorage sync prime for CII risk scores
getCachedScores() is called synchronously by country-intel.ts as a
fallback during learning mode. Without localStorage priming, the
breaker's async IndexedDB hydration hasn't run yet and returns null.
- Add shape validator (isValidCiiEntry) for untrusted localStorage data
- Add loadFromStorage/saveToStorage with 24h staleness ceiling
- Prime breaker synchronously at module load from localStorage
- Skip priming for empty cii arrays to avoid cached-empty trap
- Save to localStorage on both bootstrap and RPC success paths
Add circuit breaker + IndexedDB persistence + bootstrap hydration to
theater posture fetching — the only major panel without these resilience
layers. Replaces fragile in-memory cache (15-min TTL) and destructive
localStorage (30-min hard-delete) with the standard 3-tier pattern used
by all other panels.
* enhance supply chain panel
* fix(supply-chain): resolve P1 threat zeroing and P2 geo-first misclassification
P1: threat baseline is now always applied regardless of config
staleness — stale config only adds a review-recommended note,
never zeros the score.
P2: resolveChokepointId now checks text evidence first and only
falls back to proximity when text has no confident match.
Adds regression test: text "Bab el-Mandeb" with location near
Suez correctly resolves to bab_el_mandeb.
---------
Co-authored-by: fayez bast <fayezbast15@gmail.com>
