Commit Graph

7 Commits

Author SHA1 Message Date
Elie Habib
64c906a406 feat(eia): gold-standard /api/eia/petroleum (Railway seed → Redis → Vercel reads only) (#3161)
* feat(eia): move /api/eia/petroleum to gold-standard (Railway seed → Redis → Vercel reads only)

Live api.eia.gov fetches from the Vercel edge function were causing
FUNCTION_INVOCATION_TIMEOUT 504s on /api/eia/petroleum (Sydney edge →
US origin with no timeout, no cache, no stale fallback — one EIA blip
blew the 25s budget).

- New seeder scripts/seed-eia-petroleum.mjs — fetches WTI/Brent/
  production/inventory from api.eia.gov with per-fetch 15s timeouts,
  writes energy:eia-petroleum:v1 with the {_seed, data} envelope.
  Accepts 1-of-4 series; 0-of-4 routes to contract-mode RETRY so
  seed-meta stays stale and the bundle retries on next cron.
- Bundled into seed-bundle-energy-sources.mjs (daily, 90s timeout) —
  no new Railway service needed.
- Rewrote api/eia/[[...path]].js as a Redis-only reader via
  readJsonFromUpstash. Same response shape for backward compat with
  widgets/MCP/external callers. 503 + Retry-After on miss (never 504).
- Registered eiaPetroleum in api/health.js STANDALONE_KEYS + gated as
  ON_DEMAND_KEYS for the deploy window; promote to SEED_META
  (maxStaleMin: 4320) in a follow-up after ~7 days of clean cron.
- Tests: 14 seeder unit tests + 9 edge handler tests.

Audit result: /api/eia/petroleum was the only Vercel route fetching
dashboard data live. Every other fetch(https://…) in api/ is
auth/payments/notifications/user-initiated enrichment.

* fix(eia): close silent-stale window — add SEED_META + seed-health registration

Review finding on PR #3161: without a SEED_META entry, readSeedMeta
returns seedStale: null and classifyKey never reaches STALE_SEED.
That meant a broken Railway cron or missing EIA_API_KEY after the first
successful seed would keep /api/eia/petroleum serving stale data for
up to 7 days (TTL) while /api/health reported OK.

- api/health.js: add SEED_META.eiaPetroleum with maxStaleMin=4320
  (72h = 3× daily bundle cadence). Keep eiaPetroleum in ON_DEMAND_KEYS
  so the Vercel-instant / Railway-delayed deploy window doesn't CRIT
  on first seed, but stale-after-seed now properly fires STALE_SEED.
- api/seed-health.js: register energy:eia-petroleum in SEED_DOMAINS
  (intervalMin=1440) so the secondary health endpoint reports it too.
- Updated ON_DEMAND_KEYS comment to reflect freshness is now enforced.
2026-04-18 14:40:00 +04:00
Elie Habib
a9224254a5 fix: security hardening — CORS, auth bypass, origin validation & bump v2.2.7
- Tighten CORS regex to block worldmonitorEVIL.vercel.app spoofing
- Move sidecar /api/local-env-update behind token auth + add key allowlist
- Add postMessage origin/source validation in LiveNewsPanel
- Replace postMessage wildcard '*' targetOrigin with specific origin
- Add isDisallowedOrigin() check to 25 API endpoints missing it
- Migrate gdelt-geo & EIA from custom CORS to shared _cors.js
- Add CORS to firms-fires, stock-index, youtube/live endpoints
- Tighten youtube/embed.js ALLOWED_ORIGINS regex
- Remove 'unsafe-inline' from CSP script-src
- Add iframe sandbox attribute to YouTube embed
- Validate meta-tags URL query params with regex allowlist
2026-02-15 20:33:20 +04:00
Elie Habib
c353cf2070 Reduce egress costs, add PWA support, fix Polymarket and Railway relay
Egress optimization:
- Add s-maxage + stale-while-revalidate to all API endpoints for Vercel CDN caching
- Add vercel.json with immutable caching for hashed assets
- Add gzip compression to sidecar responses >1KB
- Add gzip to Railway RSS responses (4 paths previously uncompressed)
- Increase polling intervals: markets/crypto 60s→120s, ETF/macro/stablecoins 60s→180s
- Remove hardcoded Railway URL from theater-posture.js (now env-var only)

PWA / Service Worker:
- Add vite-plugin-pwa with autoUpdate strategy
- Cache map tiles (CacheFirst), fonts (StaleWhileRevalidate), static assets
- NetworkOnly for all /api/* routes (real-time data must be fresh)
- Manual SW registration (web only, skip Tauri)
- Add offline fallback page
- Replace manual manifest with plugin-generated manifest

Polymarket fix:
- Route dev proxy through production Vercel (bypasses JA3 blocking)
- Add 4th fallback tier: production URL as absolute fallback

Desktop/Sidecar:
- Dual-backend cache (_upstash-cache.js): Redis cloud + in-memory+file desktop
- Settings window OK/Cancel redesign
- Runtime config and secret injection improvements
2026-02-14 19:53:04 +04:00
Elie Habib
f56d44f2f2 Use wildcard CORS for *.worldmonitor.app subdomains 2026-01-23 08:30:15 +04:00
Elie Habib
81c538255d Add tech.worldmonitor.app to API CORS allowlists 2026-01-23 08:29:33 +04:00
Elie Habib
7ecb1b1597 Security hardening for EIA and USASpending features
Fixes identified by red-team audit:

EIA API Proxy:
- Restrict CORS to allowed origins only (HIGH)
- Add HTTP method validation - GET/OPTIONS only (MEDIUM)
- Remove error message information leakage (HIGH)

USASpending Service:
- Add input validation bounds for daysBack (1-90) and limit (1-50)

EconomicPanel:
- Escape all dynamic values in templates (XSS prevention)
- Escape numeric values, trend colors, icons, dates
2026-01-16 16:18:41 +04:00
Elie Habib
5bbe126484 Fix EIA API routing with Vercel catch-all route 2026-01-16 15:41:48 +04:00