Commit Graph

162 Commits

Author SHA1 Message Date
Sebastien Melki
58e42aadf9 chore(api): enforce sebuf contract + migrate drifting endpoints (#3207) (#3242)
* chore(api): enforce sebuf contract via exceptions manifest (#3207)

Adds api/api-route-exceptions.json as the single source of truth for
non-proto /api/ endpoints, with scripts/enforce-sebuf-api-contract.mjs
gating every PR via npm run lint:api-contract. Fixes the root-only blind
spot in the prior allowlist (tests/edge-functions.test.mjs), which only
scanned top-level *.js files and missed nested paths and .ts endpoints —
the gap that let api/supply-chain/v1/country-products.ts and friends
drift under proto domain URL prefixes unchallenged.

Checks both directions: every api/<domain>/v<N>/[rpc].ts must pair with
a generated service_server.ts (so a deleted proto fails CI), and every
generated service must have an HTTP gateway (no orphaned generated code).

Manifest entries require category + reason + owner, with removal_issue
mandatory for temporary categories (deferred, migration-pending) and
forbidden for permanent ones. .github/CODEOWNERS pins the manifest to
@SebastienMelki so new exceptions don't slip through review.

The manifest only shrinks: migration-pending entries (19 today) will be
removed as subsequent commits in this PR land each migration.

* refactor(maritime): migrate /api/ais-snapshot → maritime/v1.GetVesselSnapshot (#3207)

The proto VesselSnapshot was carrying density + disruptions but the frontend
also needed sequence, relay status, and candidate_reports to drive the
position-callback system. Those only lived on the raw relay passthrough, so
the client had to keep hitting /api/ais-snapshot whenever callbacks were
registered and fall back to the proto RPC only when the relay URL was gone.

This commit pushes all three missing fields through the proto contract and
collapses the dual-fetch-path into one proto client call.

Proto changes (proto/worldmonitor/maritime/v1/):
  - VesselSnapshot gains sequence, status, candidate_reports.
  - GetVesselSnapshotRequest gains include_candidates (query: include_candidates).

Handler (server/worldmonitor/maritime/v1/get-vessel-snapshot.ts):
  - Forwards include_candidates to ?candidates=... on the relay.
  - Separate 5-min in-memory caches for the candidates=on and candidates=off
    variants; they have very different payload sizes and should not share a slot.
  - Per-request in-flight dedup preserved per-variant.

Frontend (src/services/maritime/index.ts):
  - fetchSnapshotPayload now calls MaritimeServiceClient.getVesselSnapshot
    directly with includeCandidates threaded through. The raw-relay path,
    SNAPSHOT_PROXY_URL, DIRECT_RAILWAY_SNAPSHOT_URL and LOCAL_SNAPSHOT_FALLBACK
    are gone — production already routed via Vercel, the "direct" branch only
    ever fired on localhost, and the proto gateway covers both.
  - New toLegacyCandidateReport helper mirrors toDensityZone/toDisruptionEvent.

api/ais-snapshot.js deleted; manifest entry removed. Only reduced the codegen
scope to worldmonitor.maritime.v1 (buf generate --path) — regenerating the
full tree drops // @ts-nocheck from every client/server file and surfaces
pre-existing type errors across 30+ unrelated services, which is not in
scope for this PR.

Shape-diff vs legacy payload:
  - disruptions / density: proto carries the same fields, just with the
    GeoCoordinates wrapper and enum strings (remapped client-side via
    existing toDisruptionEvent / toDensityZone helpers).
  - sequence, status.{connected,vessels,messages}: now populated from the
    proto response — was hardcoded to 0/false in the prior proto fallback.
  - candidateReports: same shape; optional numeric fields come through as
    0 instead of undefined, which the legacy consumer already handled.

* refactor(sanctions): migrate /api/sanctions-entity-search → LookupSanctionEntity (#3207)

The proto docstring already claimed "OFAC + OpenSanctions" coverage but the
handler only fuzzy-matched a local OFAC Redis index — narrower than the
legacy /api/sanctions-entity-search, which proxied OpenSanctions live (the
source advertised in docs/api-proxies.mdx). Deleting the legacy without
expanding the handler would have been a silent coverage regression for
external consumers.

Handler changes (server/worldmonitor/sanctions/v1/lookup-entity.ts):
  - Primary path: live search against api.opensanctions.org/search/default
    with an 8s timeout and the same User-Agent the legacy edge fn used.
  - Fallback path: the existing OFAC local fuzzy match, kept intact for when
    OpenSanctions is unreachable / rate-limiting.
  - Response source field flips between 'opensanctions' (happy path) and
    'ofac' (fallback) so clients can tell which index answered.
  - Query validation tightened: rejects q > 200 chars (matches legacy cap).

Rate limiting:
  - Added /api/sanctions/v1/lookup-entity to ENDPOINT_RATE_POLICIES at 30/min
    per IP — matches the legacy createIpRateLimiter budget. The gateway
    already enforces per-endpoint policies via checkEndpointRateLimit.

Docs:
  - docs/api-proxies.mdx — dropped the /api/sanctions-entity-search row
    (plus the orphaned /api/ais-snapshot row left over from the previous
    commit in this PR).
  - docs/panels/sanctions-pressure.mdx — points at the new RPC URL and
    describes the OpenSanctions-primary / OFAC-fallback semantics.

api/sanctions-entity-search.js deleted; manifest entry removed.

* refactor(military): migrate /api/military-flights → ListMilitaryFlights (#3207)

Legacy /api/military-flights read a pre-baked Redis blob written by the
seed-military-flights cron and returned flights in a flat app-friendly
shape (lat/lon, lowercase enums, lastSeenMs). The proto RPC takes a bbox,
fetches OpenSky live, classifies server-side, and returns nested
GeoCoordinates + MILITARY_*_TYPE_* enum strings + lastSeenAt — same data,
different contract.

fetchFromRedis in src/services/military-flights.ts was doing nothing
sebuf-aware. Renamed it to fetchViaProto and rewrote to:

  - Instantiate MilitaryServiceClient against getRpcBaseUrl().
  - Iterate MILITARY_QUERY_REGIONS (PACIFIC + WESTERN) in parallel — same
    regions the desktop OpenSky path and the seed cron already use, so
    dashboard coverage tracks the analytic pipeline.
  - Dedup by hexCode across regions.
  - Map proto → app shape via new mapProtoFlight helper plus three reverse
    enum maps (AIRCRAFT_TYPE_REVERSE, OPERATOR_REVERSE, CONFIDENCE_REVERSE).

The seed cron (scripts/seed-military-flights.mjs) stays put: it feeds
regional-snapshot mobility, cross-source signals, correlation, and the
health freshness check (api/health.js: 'military:flights:v1'). None of
those read the legacy HTTP endpoint; they read the Redis key directly.
The proto handler uses its own per-bbox cache keys under the same prefix,
so dashboard traffic no longer races the seed cron's blob — the two paths
diverge by a small refresh lag, which is acceptable.

Docs: dropped the /api/military-flights row from docs/api-proxies.mdx.

api/military-flights.js deleted; manifest entry removed.

Shape-diff vs legacy:
  - f.location.{latitude,longitude} → f.lat, f.lon
  - f.aircraftType: MILITARY_AIRCRAFT_TYPE_TANKER → 'tanker' via reverse map
  - f.operator: MILITARY_OPERATOR_USAF → 'usaf' via reverse map
  - f.confidence: MILITARY_CONFIDENCE_LOW → 'low' via reverse map
  - f.lastSeenAt (number) → f.lastSeen (Date)
  - f.enrichment → f.enriched (with field renames)
  - Extra fields registration / aircraftModel / origin / destination /
    firstSeenAt now flow through where proto populates them.

* fix(supply-chain): thread includeCandidates through chokepoint status (#3207)

Caught by tsconfig.api.json typecheck in the pre-push hook (not covered
by the plain tsc --noEmit run that ran before I pushed the ais-snapshot
commit). The chokepoint status handler calls getVesselSnapshot internally
with a static no-auth request — now required to include the new
includeCandidates bool from the proto extension.

Passing false: server-internal callers don't need per-vessel reports.

* test(maritime): update getVesselSnapshot cache assertions (#3207)

The ais-snapshot migration replaced the single cachedSnapshot/cacheTimestamp
pair with a per-variant cache so candidates-on and candidates-off payloads
don't evict each other. Pre-push hook surfaced that tests/server-handlers
still asserted the old variable names. Rewriting the assertions to match
the new shape while preserving the invariants they actually guard:

  - Freshness check against slot TTL.
  - Cache read before relay call.
  - Per-slot in-flight dedup.
  - Stale-serve on relay failure (result ?? slot.snapshot).

* chore(proto): restore // @ts-nocheck on regenerated maritime files (#3207)

I ran 'buf generate --path worldmonitor/maritime/v1' to scope the proto
regen to the one service I was changing (to avoid the toolchain drift
that drops @ts-nocheck from 60+ unrelated files — separate issue). But
the repo convention is the 'make generate' target, which runs buf and
then sed-prepends '// @ts-nocheck' to every generated .ts file. My
scoped command skipped the sed step. The proto-check CI enforces the
sed output, so the two maritime files need the directive restored.

* refactor(enrichment): decomm /api/enrichment/{company,signals} legacy edge fns (#3207)

Both endpoints were already ported to IntelligenceService:
  - getCompanyEnrichment  (/api/intelligence/v1/get-company-enrichment)
  - listCompanySignals    (/api/intelligence/v1/list-company-signals)

No frontend callers of the legacy /api/enrichment/* paths exist. Removes:
  - api/enrichment/company.js, signals.js, _domain.js
  - api-route-exceptions.json migration-pending entries (58 remain)
  - docs/api-proxies.mdx rows for /api/enrichment/{company,signals}
  - docs/architecture.mdx reference updated to the IntelligenceService RPCs

Verified: typecheck, typecheck:api, lint:api-contract (89 files / 58 entries),
lint:boundaries, tests/edge-functions.test.mjs (136 pass),
tests/enrichment-caching.test.mjs (14 pass — still guards the intelligence/v1
handlers), make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(leads): migrate /api/{contact,register-interest} → LeadsService (#3207)

New leads/v1 sebuf service with two POST RPCs:
  - SubmitContact    → /api/leads/v1/submit-contact
  - RegisterInterest → /api/leads/v1/register-interest

Handler logic ported 1:1 from api/contact.js + api/register-interest.js:
  - Turnstile verification (desktop sources bypass, preserved)
  - Honeypot (website field) silently accepts without upstream calls
  - Free-email-domain gate on SubmitContact (422 ApiError)
  - validateEmail (disposable/offensive/typo-TLD/MX) on RegisterInterest
  - Convex writes via ConvexHttpClient (contactMessages:submit, registerInterest:register)
  - Resend notification + confirmation emails (HTML templates unchanged)

Shared helpers moved to server/_shared/:
  - turnstile.ts (getClientIp + verifyTurnstile)
  - email-validation.ts (disposable/offensive/MX checks)

Rate limits preserved via ENDPOINT_RATE_POLICIES:
  - submit-contact:    3/hour per IP (was in-memory 3/hr)
  - register-interest: 5/hour per IP (was in-memory 5/hr; desktop
    sources previously capped at 2/hr via shared in-memory map —
    now 5/hr like everyone else, accepting the small regression in
    exchange for Upstash-backed global limiting)

Callers updated:
  - pro-test/src/App.tsx contact form → new submit-contact path
  - src-tauri/sidecar/local-api-server.mjs cloud-fallback rewrites
    /api/register-interest → /api/leads/v1/register-interest when
    proxying; keeps local path for older desktop builds
  - src/services/runtime.ts isKeyFreeApiTarget allows both old and
    new paths through the WORLDMONITOR_API_KEY-optional gate

Tests:
  - tests/contact-handler.test.mjs rewritten to call submitContact
    handler directly; asserts on ValidationError / ApiError
  - tests/email-validation.test.mjs + tests/turnstile.test.mjs
    point at the new server/_shared/ modules

Deleted: api/contact.js, api/register-interest.js, api/_ip-rate-limit.js,
api/_turnstile.js, api/_email-validation.js, api/_turnstile.test.mjs.
Manifest entries removed (58 → 56). Docs updated (api-platform,
api-commerce, usage-rate-limits).

Verified: npm run typecheck + typecheck:api + lint:api-contract
(88 files / 56 entries) + lint:boundaries pass; full test:data
(5852 tests) passes; make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(pro-test): rebuild bundle for leads/v1 contact form (#3207)

Updates the enterprise contact form to POST to /api/leads/v1/submit-contact
(old path /api/contact removed in the previous commit).

Bundle is rebuilt from pro-test/src/App.tsx source change in 9ccd309d.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): address HIGH review findings 1-3 (#3207)

Three review findings from @koala73 on the sebuf-migration PR, all
silent bugs that would have shipped to prod:

### 1. Sanctions rate-limit policy was dead code

ENDPOINT_RATE_POLICIES keyed the 30/min budget under
/api/sanctions/v1/lookup-entity, but the generated route (from the
proto RPC LookupSanctionEntity) is /api/sanctions/v1/lookup-sanction-entity.
hasEndpointRatePolicy / getEndpointRatelimit are exact-string pathname
lookups, so the mismatch meant the endpoint fell through to the
generic 600/min global limiter instead of the advertised 30/min.

Net effect: the live OpenSanctions proxy endpoint (unauthenticated,
external upstream) had 20x the intended rate budget. Fixed by renaming
the policy key to match the generated route.

### 2. Lost stale-seed fallback on military-flights

Legacy api/military-flights.js cascaded military:flights:v1 →
military:flights:stale:v1 before returning empty. The new proto
handler went straight to live OpenSky/relay and returned null on miss.

Relay or OpenSky hiccup used to serve stale seeded data (24h TTL);
under the new handler it showed an empty map. Both keys are still
written by scripts/seed-military-flights.mjs on every run — fix just
reads the stale key when the live fetch returns null, converts the
seed's app-shape flights (flat lat/lon, lowercase enums, lastSeenMs)
to the proto shape (nested GeoCoordinates, enum strings, lastSeenAt),
and filters to the request bbox.

Read via getRawJson (unprefixed) to match the seed cron's writes,
which bypass the env-prefix system.

### 3. Hex-code casing mismatch broke getFlightByHex

The seed cron writes hexCode: icao24.toUpperCase() (uppercase);
src/services/military-flights.ts:getFlightByHex uppercases the lookup
input: f.hexCode === hexCode.toUpperCase(). The new proto handler
preserved OpenSky's lowercase icao24, and mapProtoFlight is a
pass-through. getFlightByHex was silently returning undefined for
every call after the migration.

Fix: uppercase in the proto handler (live + stale paths), and document
the invariant in a comment on MilitaryFlight.hex_code in
military_flight.proto so future handlers don't re-break it.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries) / lint:boundaries clean
- tests/edge-functions.test.mjs 130 pass
- make generate zero-diff (openapi spec regenerated for proto comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): restore desktop 2/hr rate cap on register-interest (#3207)

Addresses HIGH review finding #4 from @koala73. The legacy
api/register-interest.js applied a nested 2/hr per-IP cap when
`source === 'desktop-settings'`, on top of the generic 5/hr endpoint
budget. The sebuf migration lost this — desktop-source requests now
enjoy the full 5/hr cap.

Since `source` is an unsigned client-supplied field, anyone sending
`source: 'desktop-settings'` skips Turnstile AND gets 5/hr. Without
the tighter cap the Turnstile bypass is cheaper to abuse.

Added `checkScopedRateLimit` to `server/_shared/rate-limit.ts` — a
reusable second-stage Upstash limiter keyed on an opaque scope string
+ caller identifier. Fail-open on Redis errors to match existing
checkRateLimit / checkEndpointRateLimit semantics. Handlers that need
per-subscope caps on top of the gateway-level endpoint budget use this
helper.

In register-interest: when `isDesktopSource`, call checkScopedRateLimit
with scope `/api/leads/v1/register-interest#desktop`, limit=2, window=1h,
IP as identifier. On exceeded → throw ApiError(429).

### What this does not fix

This caps the blast radius of the Turnstile bypass but does not close
it — an attacker sending `source: 'desktop-settings'` still skips
Turnstile (just at 2/hr instead of 5/hr). The proper fix is a signed
desktop-secret header that authenticates the bypass; filed as
follow-up #3252. That requires coordinated Tauri build + Vercel env
changes out of scope for #3207.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries)
- tests/edge-functions.test.mjs + contact-handler.test.mjs (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): MEDIUM + LOW + rate-limit-policy CI check (#3207)

Closes out the remaining @koala73 review findings from #3242 that
didn't already land in the HIGH-fix commits, plus the requested CI
check that would have caught HIGH #1 (dead-code policy key) at
review time.

### MEDIUM #5 — Turnstile missing-secret policy default

Flip `verifyTurnstile`'s default `missingSecretPolicy` from `'allow'`
to `'allow-in-development'`. Dev with no secret = pass (expected
local); prod with no secret = reject + log. submit-contact was
already explicitly overriding to `'allow-in-development'`;
register-interest was silently getting `'allow'`. Safe default now
means a future missing-secret misconfiguration in prod gets caught
instead of silently letting bots through. Removed the now-redundant
override in submit-contact.

### MEDIUM #6 — Silent enum fallbacks in maritime client

`toDisruptionEvent` mapped `AIS_DISRUPTION_TYPE_UNSPECIFIED` / unknown
enum values → `gap_spike` / `low` silently. Refactored to return null
when either enum is unknown; caller filters nulls out of the array.
Handler doesn't produce UNSPECIFIED today, but the `gap_spike`
default would have mislabeled the first new enum value the proto
ever adds — dropping unknowns is safer than shipping wrong labels.

### LOW — Copy drift in register-interest email

Email template hardcoded `435+ Sources`; PR #3241 bumped marketing to
`500+`. Bumped in the rewritten file to stay consistent.

The `as any` on Convex mutation names carried over from legacy and
filed as follow-up #3253.

### Rate-limit-policy coverage lint

`scripts/enforce-rate-limit-policies.mjs` validates every key in
`ENDPOINT_RATE_POLICIES` resolves to a proto-generated gateway route
by cross-referencing `docs/api/*.openapi.yaml`. Fails with the
sanctions-entity-search incident referenced in the error message so
future drift has a paper trail.

Wired into package.json (`lint:rate-limit-policies`) and the pre-push
hook alongside `lint:boundaries`. Smoke-tested both directions —
clean repo passes (5 policies / 175 routes), seeded drift (the exact
HIGH #1 typo) fails with the advertised remedy text.

### Verified
- `lint:rate-limit-policies` ✓
- `typecheck` + `typecheck:api` ✓
- `lint:api-contract` ✓ (56 entries)
- `lint:boundaries` ✓
- edge-functions + contact-handler tests (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 5): decomm /api/eia/* + migrate /api/satellites → IntelligenceService (#3207)

Both targets turned out to be decomm-not-migration cases. The original
plan called for two new services (economic/v1.GetEiaSeries +
natural/v1.ListSatellitePositions) but research found neither was
needed:

### /api/eia/[[...path]].js — pure decomm, zero consumers

The "catch-all" is a misnomer — only two paths actually worked,
/api/eia/health and /api/eia/petroleum, both Redis-only readers.
Zero frontend callers in src/. Zero server-side readers. Nothing
consumes the `energy:eia-petroleum:v1` key that seed-eia-petroleum.mjs
writes daily.

The EIA data the frontend actually uses goes through existing typed
RPCs in economic/v1: GetEnergyPrices, GetCrudeInventories,
GetNatGasStorage, GetEnergyCapacity. None of those touch /api/eia/*.

Building GetEiaSeries would have been dead code. Deleted the legacy
file + its test (tests/api-eia-petroleum.test.mjs — it only covered
the legacy endpoint, no behavior to preserve). Empty api/eia/ dir
removed.

**Note for review:** the Redis seed cron keeps running daily and
nothing consumes it. If that stays unused, seed-eia-petroleum.mjs
should be retired too (separate PR). Out of scope for sebuf-migration.

### /api/satellites.js — Learning #2 strikes again

IntelligenceService.ListSatellites already exists at
/api/intelligence/v1/list-satellites, reads the same Redis key
(intelligence:satellites:tle:v1), and supports an optional country
filter the legacy didn't have.

One frontend caller in src/services/satellites.ts needed to switch
from `fetch(toApiUrl('/api/satellites'))` to the typed
IntelligenceServiceClient.listSatellites. Shape diff was tiny —
legacy `noradId` became proto `id` (handler line 36 already picks
either), everything else identical. alt/velocity/inclination in the
proto are ignored by the caller since it propagates positions
client-side via satellite.js.

Kept the client-side cache + failure cooldown + 20s timeout (still
valid concerns at the caller level).

### Manifest + docs
- api-route-exceptions.json: 56 → 54 entries (both removed)
- docs/api-proxies.mdx: dropped the two rows from the Raw-data
  passthroughs table

### Verified
- typecheck + typecheck:api ✓
- lint:api-contract (54 entries) / lint:boundaries / lint:rate-limit-policies ✓
- tests/edge-functions.test.mjs 127 pass (down from 130 — 3 tests were
  for the deleted eia endpoint)
- make generate zero-diff (no proto changes)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 6): migrate /api/supply-chain/v1/{country-products,multi-sector-cost-shock} → SupplyChainService (#3207)

Both endpoints were hand-rolled TS handlers sitting under a proto URL prefix —
the exact drift the manifest guardrail flagged. Promoted both to typed RPCs:

- GetCountryProducts → /api/supply-chain/v1/get-country-products
- GetMultiSectorCostShock → /api/supply-chain/v1/get-multi-sector-cost-shock

Handlers preserve the existing semantics: PRO-gate via isCallerPremium(ctx.request),
iso2 / chokepointId validation, raw bilateral-hs4 Redis read (skip env-prefix to
match seeder writes), CHOKEPOINT_STATUS_KEY for war-risk tier, and the math from
_multi-sector-shock.ts unchanged. Empty-data and non-PRO paths return the typed
empty payload (no 403 — the sebuf gateway pattern is empty-payload-on-deny).

Client wrapper switches from premiumFetch to client.getCountryProducts/
client.getMultiSectorCostShock. Legacy MultiSectorShock / MultiSectorShockResponse /
CountryProductsResponse names remain as type aliases of the generated proto types
so CountryBriefPanel + CountryDeepDivePanel callsites compile with zero churn.

Manifest 54 → 52. Rate-limit gateway routes 175 → 177.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(gateway): add cache-tier entries for new supply-chain RPCs (#3207)

Pre-push tests/route-cache-tier.test.mjs caught the missing entries.
Both PRO-gated, request-varying — match the existing supply-chain PRO cohort
(get-country-cost-shock, get-bypass-options, etc.) at slow-browser tier.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 7): migrate /api/scenario/v1/{run,status,templates} → ScenarioService (#3207)

Promote the three literal-filename scenario endpoints to a typed sebuf
service with three RPCs:

  POST /api/scenario/v1/run-scenario        (RunScenario)
  GET  /api/scenario/v1/get-scenario-status (GetScenarioStatus)
  GET  /api/scenario/v1/list-scenario-templates (ListScenarioTemplates)

Preserves all security invariants from the legacy handlers:
- 405 for wrong method (sebuf service-config method gate)
- scenarioId validation against SCENARIO_TEMPLATES registry
- iso2 regex ^[A-Z]{2}$
- JOB_ID_RE path-traversal guard on status
- Per-IP 10/min rate limit (moved to gateway ENDPOINT_RATE_POLICIES)
- Queue-depth backpressure (>100 → 429)
- PRO gating via isCallerPremium
- AbortSignal.timeout on every Redis pipeline (runRedisPipeline helper)

Wire-level diffs vs legacy:
- Per-user RL now enforced at the gateway (same 10/min/IP budget).
- Rate-limit response omits Retry-After header; retryAfter is in the
  body per error-mapper.ts convention.
- ListScenarioTemplates emits affectedHs2: [] when the registry entry
  is null (all-sectors sentinel); proto repeated cannot carry null.
- RunScenario returns { jobId, status } (no statusUrl field — unused
  by SupplyChainPanel, drop from wire).

Gateway wiring:
- server/gateway.ts RPC_CACHE_TIER: list-scenario-templates → 'daily'
  (matches legacy max-age=3600); get-scenario-status → 'slow-browser'
  (premium short-circuit target, explicit entry required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: swap old run/status for the new
  run-scenario/get-scenario-status paths.
- api/scenario/v1/{run,status,templates}.ts deleted; 3 manifest
  exceptions removed (63 → 52 → 49 migration-pending).

Client:
- src/services/scenario/index.ts — typed client wrapper using
  premiumFetch (injects Clerk bearer / API key).
- src/components/SupplyChainPanel.ts — polling loop swapped from
  premiumFetch strings to runScenario/getScenarioStatus. Hard 20s
  timeout on run preserved via AbortSignal.any.

Tests:
- tests/scenario-handler.test.mjs — 18 new handler-level tests
  covering every security invariant + the worker envelope coercion.
- tests/edge-functions.test.mjs — scenario sections removed,
  replaced with a breadcrumb pointer to the new test file.

Docs: api-scenarios.mdx, scenario-engine.mdx, usage-rate-limits.mdx,
usage-errors.mdx, supply-chain.mdx refreshed with new paths.

Verified: typecheck, typecheck:api, lint:api-contract (49 entries),
lint:rate-limit-policies (6/180), lint:boundaries, route-cache-tier
(parity), full edge-functions (117) + scenario-handler (18).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 8): migrate /api/v2/shipping/{route-intelligence,webhooks} → ShippingV2Service (#3207)

Partner-facing endpoints promoted to a typed sebuf service. Wire shape
preserved byte-for-byte (camelCase field names, ISO-8601 fetchedAt, the
same subscriberId/secret formats, the same SET + SADD + EXPIRE 30-day
Redis pipeline). Partner URLs /api/v2/shipping/* are unchanged.

RPCs landed:
- GET  /route-intelligence  → RouteIntelligence  (PRO, slow-browser)
- POST /webhooks            → RegisterWebhook    (PRO)
- GET  /webhooks            → ListWebhooks       (PRO, slow-browser)

The existing path-parameter URLs remain on the legacy edge-function
layout because sebuf's HTTP annotations don't currently model path
params (grep proto/**/*.proto for `path: "{…}"` returns zero). Those
endpoints are split into two Vercel dynamic-route files under
api/v2/shipping/webhooks/, behaviorally identical to the previous
hybrid file but cleanly separated:
- GET  /webhooks/{subscriberId}                → [subscriberId].ts
- POST /webhooks/{subscriberId}/rotate-secret  → [subscriberId]/[action].ts
- POST /webhooks/{subscriberId}/reactivate     → [subscriberId]/[action].ts

Both get manifest entries under `migration-pending` pointing at #3207.

Other changes
- scripts/enforce-sebuf-api-contract.mjs: extended GATEWAY_RE to accept
  api/v{N}/{domain}/[rpc].ts (version-first) alongside the canonical
  api/{domain}/v{N}/[rpc].ts; first-use of the reversed ordering is
  shipping/v2 because that's the partner contract.
- vite.config.ts: dev-server sebuf interceptor regex extended to match
  both layouts; shipping/v2 import + allRoutes entry added.
- server/gateway.ts: RPC_CACHE_TIER entries for /api/v2/shipping/
  route-intelligence + /webhooks (slow-browser; premium-gated endpoints
  short-circuit to slow-browser but the entries are required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: route-intelligence + webhooks added.
- tests/shipping-v2-handler.test.mjs: 18 handler-level tests covering
  PRO gate, iso2/cargoType/hs2 coercion, SSRF guards (http://, RFC1918,
  cloud metadata, IMDS), chokepoint whitelist, alertThreshold range,
  secret/subscriberId format, pipeline shape + 30-day TTL, cross-tenant
  owner isolation, `secret` omission from list response.

Manifest delta
- Removed: api/v2/shipping/route-intelligence.ts, api/v2/shipping/webhooks.ts
- Added:   api/v2/shipping/webhooks/[subscriberId].ts (migration-pending)
- Added:   api/v2/shipping/webhooks/[subscriberId]/[action].ts (migration-pending)
- Added:   api/internal/brief-why-matters.ts (internal-helper) — regression
  surface from the #3248 main merge, which introduced the file without a
  manifest entry. Filed here to keep the lint green; not strictly in scope
  for commit 8 but unblocking.

Net result: 49 → 47 `migration-pending` entries (one net-removal even
though webhook path-params stay pending, because two files collapsed
into two dynamic routes).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 1): SupplyChainServiceClient must use premiumFetch (#3207)

Signed-in browser pro users were silently hitting 401 on 8 supply-chain
premium endpoints (country-products, multi-sector-cost-shock,
country-chokepoint-index, bypass-options, country-cost-shock,
sector-dependency, route-explorer-lane, route-impact). The shared
client was constructed with globalThis.fetch, so no Clerk bearer or
X-WorldMonitor-Key was injected. The gateway's validateApiKey runs
with forceKey=true for PREMIUM_RPC_PATHS and 401s before isCallerPremium
is consulted. The generated client's try/catch collapses the 401 into
an empty-fallback return, leaving panels blank with no visible error.

Fix is one line at the client constructor: swap globalThis.fetch for
premiumFetch. The same pattern is already in use for insider-transactions,
stock-analysis, stock-backtest, scenario, trade (premiumClient) — this
was an omission on this client, not a new pattern.

premiumFetch no-ops safely when no credentials are available, so the
5 non-premium methods on this client (shippingRates, chokepointStatus,
chokepointHistory, criticalMinerals, shippingStress) continue to work
unchanged.

This also fixes two panels that were pre-existing latently broken on
main (chokepoint-index, bypass-options, etc. — predating #3207, not
regressions from it). Commit 6 expanded the surface by routing two more
methods through the same buggy client; this commit fixes the class.

From koala73 review (#3242 second-pass, HIGH new #1):
> Exact class PR #3233 fixed for RegionalIntelligenceBoard /
> DeductionPanel / trade / country-intel. Supply-chain was not in
> #3233's scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 2): restore 400 on input-shape errors for 2 supply-chain handlers (#3207)

Commit 6 collapsed all non-happy paths into empty-200 on
`get-country-products` and `get-multi-sector-cost-shock`, including
caller-bug cases that legacy returned 400 for:

- get-country-products: malformed iso2 → empty 200 (was 400)
- get-multi-sector-cost-shock: malformed iso2 / missing chokepointId /
  unknown chokepointId → empty 200 (was 400)

The commit message for 6 called out the 403-for-non-pro → empty-200
shift ("sebuf gateway pattern is empty-payload-on-deny") but not the
400 shift. They're different classes:

- Empty-payload-200 for PRO-deny: intentional contract change, already
  documented and applied across the service. Generated clients treat
  "you lack PRO" as "no data" — fine.
- Empty-payload-200 for malformed input: caller bug silently masked.
  External API consumers can't distinguish "bad wiring" from "genuinely
  no data", test harnesses lose the signal, bad calling code doesn't
  surface in Sentry.

Fix: `throw new ValidationError(violations)` on the 3 input-shape
branches. The generated sebuf server maps ValidationError → HTTP 400
(see src/generated/server/.../service_server.ts and leads/v1 which
already uses this pattern).

PRO-gate deny stays as empty-200 — that contract shift was intentional
and is preserved.

Regression tests added at tests/supply-chain-validation.test.mjs (8
cases) pinning the three-way contract:
- bad input                         → 400 (ValidationError)
- PRO-gate deny on valid input      → 200 empty
- valid PRO input, no data in Redis → 200 empty (unchanged)

From koala73 review (#3242 second-pass, HIGH new #2).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 3): restore statusUrl on RunScenarioResponse + document 202→200 wire break (#3207)

Commit 7 silently shifted /api/scenario/v1/run-scenario's response
contract in two ways that the commit message covered only partially:

1. HTTP 202 Accepted → HTTP 200 OK
2. Dropped `statusUrl` string from the response body

The `statusUrl` drop was mentioned as "unused by SupplyChainPanel" but
not framed as a contract change. The 202 → 200 shift was not mentioned
at all. This is a same-version (v1 → v1) migration, so external callers
that key off either signal — `response.status === 202` or
`response.body.statusUrl` — silently branch incorrectly.

Evaluated options:
  (a) sebuf per-RPC status-code config — not available. sebuf's
      HttpConfig only models `path` and `method`; no status annotation.
  (b) Bump to scenario/v2 — judged heavier than the break itself for
      a single status-code shift. No in-repo caller uses 202 or
      statusUrl; the docs-level impact is containable.
  (c) Accept the break, document explicitly, partially restore.

Took option (c):

- Restored `statusUrl` in the proto (new field `string status_url = 3`
  on RunScenarioResponse). Server computes
  `/api/scenario/v1/get-scenario-status?jobId=<encoded job_id>` and
  populates it on every successful enqueue. External callers that
  followed this URL keep working unchanged.
- 202 → 200 is not recoverable inside the sebuf generator, so it is
  called out explicitly in two places:
    - docs/api-scenarios.mdx now includes a prominent `<Warning>` block
      documenting the v1→v1 contract shift + the suggested migration
      (branch on response body shape, not HTTP status).
    - RunScenarioResponse proto comment explains why 200 is the new
      success status on enqueue.
  OpenAPI bundle regenerated to reflect the restored statusUrl field.

- Regression test added in tests/scenario-handler.test.mjs pinning
  `statusUrl` to the exact URL-encoded shape — locks the invariant so
  a future proto rename or handler refactor can't silently drop it
  again.

From koala73 review (#3242 second-pass, HIGH new #3).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 1/2): close webhook tenant-isolation gap on shipping/v2 (#3207)

Koala flagged this as a merge blocker in PR #3242 review.

server/worldmonitor/shipping/v2/{register-webhook,list-webhooks}.ts
migrated without reinstating validateApiKey(req, { forceKey: true }),
diverging from both the sibling api/v2/shipping/webhooks/[subscriberId]
routes and the documented "X-WorldMonitor-Key required" contract in
docs/api-shipping-v2.mdx.

Attack surface: the gateway accepts Clerk bearer auth as a pro signal.
A Clerk-authenticated pro user with no X-WorldMonitor-Key reaches the
handler, callerFingerprint() falls back to 'anon', and every such
caller collapses into a shared webhook:owner:anon:v1 bucket. The
defense-in-depth ownerTag !== ownerHash check in list-webhooks.ts
doesn't catch it because both sides equal 'anon' — every Clerk-session
holder could enumerate / overwrite every other Clerk-session pro
tenant's registered webhook URLs.

Fix: reinstate validateApiKey(ctx.request, { forceKey: true }) at the
top of each handler, throwing ApiError(401) when absent. Matches the
sibling routes exactly and the published partner contract.

Tests:
- tests/shipping-v2-handler.test.mjs: two existing "non-PRO → 403"
  tests for register/list were using makeCtx() with no key, which now
  fails at the 401 layer first. Renamed to "no API key → 401
  (tenant-isolation gate)" with a comment explaining the failure mode
  being tested. 18/18 pass.

Verified: typecheck:api, lint:api-contract (no change), lint:boundaries,
lint:rate-limit-policies, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 2/2): restore v1 path aliases on scenario + supply-chain (#3207)

Koala flagged this as a merge blocker in PR #3242 review.

Commits 6 + 7 of #3207 renamed five documented v1 URLs to the sebuf
method-derived paths and deleted the legacy edge-function files:

  POST /api/scenario/v1/run                       → run-scenario
  GET  /api/scenario/v1/status                    → get-scenario-status
  GET  /api/scenario/v1/templates                 → list-scenario-templates
  GET  /api/supply-chain/v1/country-products      → get-country-products
  GET  /api/supply-chain/v1/multi-sector-cost-shock → get-multi-sector-cost-shock

server/router.ts is an exact static-match table (Map keyed on `METHOD
PATH`), so any external caller — docs, partner scripts, grep-the-
internet — hitting the old documented URL would 404 on first request
after merge. Commit 8 (shipping/v2) preserved partner URLs byte-for-
byte; the scenario + supply-chain renames missed that discipline.

Fix: add five thin alias edge functions that rewrite the pathname to
the canonical sebuf path and delegate to the domain [rpc].ts gateway
via a new server/alias-rewrite.ts helper. Premium gating, rate limits,
entitlement checks, and cache-tier lookups all fire on the canonical
path — aliases are pure URL rewrites, not a duplicate handler pipeline.

  api/scenario/v1/{run,status,templates}.ts
  api/supply-chain/v1/{country-products,multi-sector-cost-shock}.ts

Vite dev parity: file-based routing at api/ is a Vercel concern, so the
dev middleware (vite.config.ts) gets a matching V1_ALIASES rewrite map
before the router dispatch.

Manifest: 5 new entries under `deferred` with removal_issue=#3282
(tracking their retirement at the next v1→v2 break). lint:api-contract
stays green (89 files checked, 55 manifest entries validated).

Docs:
- docs/api-scenarios.mdx: migration callout at the top with the full
  old→new URL table and a link to the retirement issue.
- CHANGELOG.md + docs/changelog.mdx: Changed entry documenting the
  rename + alias compat + the 202→200 shift (from commit 23c821a1).

Verified: typecheck:api, lint:api-contract, lint:rate-limit-policies,
lint:boundaries, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-22 09:55:59 +03:00
Elie Habib
50626a40c7 chore: bump version to 2.6.7 (#2637)
* chore: bump version to 2.6.7

* chore: sync Cargo.lock to 2.6.7 (was stuck at 2.6.5)

* ci: skip lint/test/typecheck for non-code PRs (docs, Tauri, version bumps)

Added paths-ignore to lint-code, test, and typecheck workflows so they
don't run when only markdown, docs, src-tauri config, or desktop build
files change. Push to main still runs unconditionally.
2026-04-03 07:37:45 +04:00
Elie Habib
5ba7523629 chore: bump version to 2.6.6 (#2601)
Forces new Sentry release tag so CSP listener fixes are tracked
separately from 2.6.5 sessions still running old code.
2026-04-01 22:51:37 +04:00
Elie Habib
a969a9e3a3 feat(auth): integrate clerk.dev (#1812)
* feat(auth): integrate better-auth with @better-auth/infra dash plugin

Wire up better-auth server config with the dash() plugin from
@better-auth/infra, and the matching sentinelClient() on the
client side. Adds BETTER_AUTH_API_KEY to .env.example.

* feat(auth): swap @better-auth/infra for @convex-dev/better-auth

[10-01 task 1] Install @convex-dev/better-auth@0.11.2, remove
@better-auth/infra, delete old server/auth.ts skeleton, rewrite
auth-client.ts to use crossDomainClient + convexClient plugins.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): create Convex auth component files

[10-01 task 2] Add convex.config.ts (register betterAuth component),
auth.config.ts (JWT/JWKS provider), auth.ts (better-auth server with
Convex adapter, crossDomain + convex plugins), http.ts (mount auth
routes with CORS). Uses better-auth/minimal for lighter bundle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): add admin, organization, and dash plugins

[10-01] Re-install @better-auth/infra for dash() plugin to enable
dash.better-auth.com admin dashboard. Add admin() and organization()
plugins from better-auth/plugins for user and org management.
Update both server (convex/auth.ts) and client (auth-client.ts).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): drop @better-auth/infra (Node.js deps incompatible with Convex V8)

Keep admin() and organization() from better-auth/plugins (V8-safe).
@better-auth/infra's dash() transitively imports SAML/SSO with
node:crypto, fs, zlib — can't run in Convex's serverless runtime.
Dashboard features available via admin plugin endpoints instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(11-01): create auth-state.ts with OTT handler and session subscription

- Add initAuthState() for OAuth one-time token verification on page load
- Add subscribeAuthState() reactive wrapper around useSession nanostore atom
- Add getAuthState() synchronous snapshot getter
- Export AuthUser and AuthSession types for UI consumption

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(11-01): add Google OAuth provider and wire initAuthState into App.ts

- Add socialProviders.google with GOOGLE_CLIENT_ID/SECRET to convex/auth.ts
- Add all variant subdomains to trustedOrigins for cross-subdomain CORS
- Call initAuthState() in App.init() before panelLayout.init()
- Add authModal field to AppContext interface (prepares for Plan 02)
- Add authModal: null to App constructor state initialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(11-02): create AuthModal with Sign In/Sign Up tabs and Google OAuth

- Sign In tab: email/password form calling authClient.signIn.email()
- Sign Up tab: name/email/password form calling authClient.signUp.email()
- Google OAuth button calling authClient.signIn.social({ provider: 'google', callbackURL: '/' })
- Auto-close on successful auth via subscribeAuthState() subscription
- Escape key, overlay click, and X button close the modal
- Loading states, error display, and client-side validation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(11-02): add AuthHeaderWidget, mount in header, add auth CSS

- AuthHeaderWidget: reactive header widget showing Sign In button (anonymous) or avatar + dropdown (authenticated)
- User dropdown: name, email, Free tier badge, Sign Out button calling authClient.signOut()
- setupAuthWidget() in EventHandlerManager creates modal + widget, mounts at authWidgetMount span
- authWidgetMount added to panel-layout.ts header-right, positioned before download wrapper
- setupAuthWidget() called from App.ts after setupUnifiedSettings()
- Full auth CSS: modal styles, tabs, forms, Google button, header widget, avatar, dropdown

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(11-02): add localhost:3000 to trustedOrigins for local dev CORS

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): remove admin/organization plugins that break Convex adapter validator

The admin() plugin adds banned/role fields to user creation data, but the
@convex-dev/better-auth adapter validator doesn't include them. These plugins
are Phase 12 work — will re-add with additionalFields config when needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-01): add Resend email transport, verification + reset callbacks, role field

- Install resend SDK for transactional email
- Add emailVerification with sendOnSignUp:true and fire-and-forget Resend callbacks
- Add sendResetPassword callback with 1-hour token expiry
- Add user.additionalFields.role (free/pro, input:false, defaultValue:free)
- Create userRoles fallback table in schema with by_userId index
- Create getUserRole query and setUserRole mutation in convex/userRoles.ts
- Lazy-init Resend client to avoid Convex module analysis error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-01): enhance auth-state with emailVerified and role fields

- Add emailVerified (boolean) and role ('free' | 'pro') to AuthUser interface
- Fetch role from Convex userRoles table via HTTP query after session hydration
- Cache role per userId to avoid redundant fetches
- Re-notify subscribers asynchronously when role is fetched for a new user
- Map emailVerified from core better-auth user field (default false)
- Derive Convex cloud URL from VITE_CONVEX_SITE_URL env var

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(12-01): add Convex generated files from deployment

- Track convex/_generated/ files produced by npx convex dev --once

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-03): create panel-gating service with auth-aware showGatedCta

- Add PanelGateReason enum (NONE/ANONYMOUS/UNVERIFIED/FREE_TIER)
- Add getPanelGateReason() computing gating from AuthSession + premium flag
- Add Panel.showGatedCta() rendering auth-aware CTA overlays
- Add Panel.unlockPanel() to reverse locked state
- Extract lockSvg to module-level const shared by showLocked/showGatedCta
- Add i18n keys: signInToUnlock, signIn, verifyEmailToUnlock, resendVerification, upgradeDesc, upgradeToPro

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-02): add forgot password flow, password reset form, and token detection

- Widen authModal interface in app-context.ts to support reset-password mode and setResetToken
- AuthModal refactored with 4 views: signin, signup, forgot-password, reset-password
- Forgot password view sends reset email via authClient.requestPasswordReset
- Reset password form validates matching passwords and calls authClient.resetPassword
- auth-state.ts detects ?token= param from email links, stores as pendingResetToken
- App.ts routes pending reset token to auth modal after UI initialization
- CSS for forgot-link, back-link, and success message elements

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-02): add email verification banner to AuthHeaderWidget and tier badge

- Show non-blocking verification banner below header for unverified users
- Banner has "Resend" button calling authClient.sendVerificationEmail
- Banner is dismissible (stored in sessionStorage, reappears next session)
- Tier badge dynamically shows Free/Pro based on user.role
- Pro badge has gradient styling distinct from Free badge
- Dropdown shows unverified status indicator with yellow dot
- Banner uses fixed positioning, does not push content down
- CSS for banner, pro badge, and verification status indicators

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-03): wire reactive auth-based gating into panel-layout

- Add WEB_PREMIUM_PANELS Set (stock-analysis, stock-backtest, daily-market-brief)
- Subscribe to auth state changes in PanelLayoutManager.init()
- Add updatePanelGating() iterating panels with getPanelGateReason()
- Add getGateAction() returning CTA callbacks per gate reason
- Remove inline showLocked() calls for web premium panels
- Preserve desktop _lockPanels for forecast, oref-sirens, telegram-intel
- Clean up auth subscription in destroy()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(13-01): create auth-token utility and inject Bearer header in web fetch redirect

- Add src/services/auth-token.ts with getSessionBearerToken() that reads session token from localStorage
- Add WEB_PREMIUM_API_PATHS Set for the 4 premium market API paths
- Inject Authorization: Bearer header in installWebApiRedirect() for premium paths when session exists
- Desktop installRuntimeFetchPatch() left unchanged (API key only)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(13-01): create server-side session validation module

- Add server/auth-session.ts with validateBearerToken() for Vercel edge gateway
- Validates tokens via Convex /api/auth/get-session with Better-Auth-Cookie header
- Falls back to userRoles:getUserRole Convex query for role resolution
- In-memory cache with 60s TTL and 100-entry cap
- Network errors not cached to allow retry on next request

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(13-02): add bearer token fallback auth for premium API endpoints

- Dynamic import of auth-session.ts when premium endpoint + API key fails
- Valid pro session tokens fall through to route handler
- Non-pro authenticated users get 403 'Pro subscription required'
- Invalid/expired tokens get 401 'Invalid or expired session'
- Non-premium endpoints and static API key flow unchanged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): sign-in button invisible in dark theme — white on white

--accent is #fff in dark theme, so background: var(--accent) + color: #fff
was invisible. Changed to transparent background with var(--text) color.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): add premium panel keys to full and finance variant configs

stock-analysis, stock-backtest, and daily-market-brief were defined in
the shared panels.ts but missing from variant DEFAULT_PANELS, causing
shouldCreatePanel() to return false and panel gating CTAs to never render.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(auth): add Playwright smoke tests for auth UI (phases 12-13)

6 tests covering: Sign In button visibility, auth modal opening,
modal views (Sign In/Sign Up/Forgot Password), premium panel gating
for anonymous users, and auth token absence when logged out.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): remove role additionalField that breaks Convex component validator

The betterAuth Convex component has a strict input validator for the
user model that doesn't include custom fields. The role additionalField
caused ArgumentValidationError on sign-up. Roles are already stored in
the separate userRoles table — no data loss.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): use Authorization Bearer header for Convex session validation

Better-Auth-Cookie header returned null — the crossDomain plugin's
get-session endpoint expects Authorization: Bearer format instead.
Confirmed via curl against live Convex deployment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): use verified worldmonitor.app domain for auth emails

Was using noreply@resend.dev (testing domain) which can't send to
external recipients. Switched to noreply@worldmonitor.app matching
existing waitlist/contact emails.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): await Resend email sends — Convex kills dangling promises

void (fire-and-forget) causes Convex to terminate the fetch before
Resend receives it. Await ensures emails actually get sent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update Convex generated auth files after config changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): guard against undefined VITE_CONVEX_SITE_URL in auth-state

The Convex cloud URL derivation crashed the entire app when
VITE_CONVEX_SITE_URL wasn't set in the build environment (Vercel
preview). Now gracefully defaults to empty string and skips role
fetching when the URL is unavailable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): add dash + organization plugins, remove Google OAuth, fix dark mode button

- Add @better-auth/infra dash plugin for hosted admin dashboard
- Add organization plugin for org management in dashboard
- Add dash.better-auth.com to trustedOrigins
- Remove Google OAuth (socialProviders, button, divider, CSS)
- Fix auth submit button invisible in dark mode (var(--accent) → #3b82f6)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): replace dash plugin with admin — @better-auth/infra incompatible with Convex V8

@better-auth/infra imports SSO/SAML libraries requiring Node.js built-ins
(crypto, fs, stream) which Convex's V8 runtime doesn't support.
Replaced with admin plugin from better-auth/plugins which provides
user management endpoints (set-role, list-users, ban, etc.) natively.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove stale Convex generated files after plugin update

Convex dev regenerated _generated/ — the per-module JS files
(auth.js, http.js, schema.js, etc.) are no longer emitted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(auth): remove organization plugin — will add in subsequent PR

Organization support (team accounts, invitations, member management)
is not wired into any frontend flow yet. Removing to keep the auth
PR focused on email/password + admin endpoints. Will add back when
building the org/team feature.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add authentication & panel gating guide

Documents the auth stack, panel gating configuration, server-side
session enforcement, environment variables, and user roles.
Includes step-by-step guide for adding new premium panels.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): stub panel-gating in RuntimeConfigPanel test harness

Panel.ts now imports @/services/panel-gating, which wasn't stubbed —
causing the real runtime.ts (with window.location) to be bundled,
breaking Node.js tests with "ReferenceError: location is not defined".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): allow Vercel preview origins in Convex trustedOrigins

* fix(auth): broaden Convex trustedOrigins to cover *.worldmonitor.app previews

* fix(auth): use hostonly wildcard pattern for *.worldmonitor.app in trustedOrigins

* fix(auth): add Convex site origins to trustedOrigins

* fix(ci): add convex/ to vercel-ignore watched paths

* fix(auth): remove admin() plugin — adds banned/role fields rejected by Convex validator

* fix(auth): remove admin() plugin — injects banned/role fields rejected by Convex betterAuth validator

* feat(auth): replace email/password with email OTP passwordless flow

- Replace emailAndPassword + emailVerification with emailOTP plugin
- Rewrite AuthModal: email entry -> OTP code verification (no passwords)
- Remove admin() plugin (caused Convex schema validation errors)
- Remove email verification banner and UNVERIFIED gate reason (OTP
  inherently verifies email)
- Remove password reset flow (forgot/reset password views, token handling)
- Clean up unused CSS (tabs, verification banner, success messages)
- Update docs to reflect new passwordless auth stack

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(quick-2): harden Convex userRoles and add role cache TTL

- P0: Convert setUserRole from mutation to internalMutation (not callable from client)
- P2: Add 5-minute TTL to role cache in auth-state.ts
- P2: Add localStorage shape warning on auth-token.ts
- P3: Document getUserRole public query trade-off
- P3: Fix misleading cache comment in auth-session.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(quick-2): auth widget teardown, E2E test rewrite, gateway comment

- P2: Store authHeaderWidget on AppContext, destroy in EventHandlerManager.destroy()
- P2: Also destroy authModal in destroy() to prevent leaked subscriptions
- P1: Rewrite E2E tests for 2-view OTP modal (email input + submit button)
- P1: Remove stale "Sign Up" and "Forgot Password" test assertions
- P2: Replace flaky waitForTimeout(5000) with Playwright auto-retry assertion
- P3: Add clarifying comment on premium bearer-token fallback in gateway

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(header): restructure header/footer, add profile editing, pro-gate playback/export

- Remove version, @eliehabib, GitHub link, and download button from header
- Move version + @eliehabib credit to footer brand line; download link to footer nav
- Move auth widget (profile avatar) to far right of header (after settings gear)
- Add default generic SVG avatar for users with no image and no name
- Add profile editing in auth dropdown: display name + avatar URL with Save/Cancel
- Add Settings shortcut in auth dropdown (opens UnifiedSettings)
- Gate Historical Playback and Export controls behind pro role (hidden for free users)
- Reactive pro-gate: subscribes to auth state changes, stores unsub in proGateUnsubscribers[]
- Clean up proGateUnsubscribers on EventHandlerManager.destroy() to prevent leaks
- Fix: render Settings button unconditionally (hidden via style), stable DOM structure
- Fix: typed updateUser call with runtime existence check instead of (any) cast
- Make initFooterDownload() private to match class conventions

* feat(analytics): add Umami auth integration and event tracking

- Wire analytics.ts facade to Umami (port from main #1914):
  search, country, map layers, panels, LLM, theme, language,
  variant switch, webcam, download, findings, deeplinks
- Add Window.umami shim to vite-env.d.ts
- Add initAuthAnalytics() that subscribes to auth state and calls
  identifyUser(id, role) / clearIdentity() on sign-in/sign-out
- Add trackSignIn, trackSignUp, trackSignOut, trackGateHit exports
- Call initAuthAnalytics() from App.ts after initAuthState()
- Track sign-in/sign-up (via isNewUser flag) in AuthModal OTP verify
- Track sign-out in AuthHeaderWidget before authClient.signOut()
- Track gate-hit for export, playback (event-handlers) and pro-banner

* feat(auth): professional avatar widget with colored initials and clean profile edit

- Replace white-circle avatar with deterministic colored initials (Gmail/Linear style)
- Avatar color derived from email hash across 8-color palette
- Dropdown redesigned: row layout with large avatar + name/email/tier info
- Profile edit form: name-only (removed avatar URL field)
- Remove Settings button from dropdown (gear icon in header is sufficient)
- Discord community widget: single CTA link, no redundant text label
- Add all missing CSS for dropdown interior, profile edit form, menu items

* fix(auth): lock down billing tier visibility and fix TOCTOU race

P1: getUserRole converted to internalQuery — billing tier no longer
accessible via any public Convex client API. Exposed only through
the new authenticated /api/user-role HTTP action which validates
the session Bearer token before returning the role.

P1: subscribeAuthState generation counter + AbortController prevents
rapid sign-in/sign-out from delivering stale role for wrong user.

P2: typed RawSessionUser/RawSessionValue interfaces replace any casts
at the better-auth nanostore boundary. fetchUserRole drops userId
param — server derives identity from Bearer token only.

P2: isNewUser heuristic removed from OTP verify — better-auth emailOTP
has no reliable isNewUser signal. All verifications tracked as
trackSignIn. OTP resend gets 30s client-side cooldown.

P2: auth-token.ts version pin comment added (better-auth@1.5.5 +
@convex-dev/better-auth@0.11.2). Gateway inner PREMIUM_RPC_PATHS
comment clarified to explain why it is not redundant.

Adds tests/auth-session.test.mts: 11 tests covering role fallback
endpoint selection, fail-closed behavior, and CORS origin matching.

* feat(quick-4): replace better-auth with Clerk JS -- packages, Convex config, browser auth layer

- Remove better-auth, @convex-dev/better-auth, @better-auth/infra, resend from dependencies
- Add @clerk/clerk-js and jose to dependencies
- Rewrite convex/auth.config.ts for Clerk issuer domain
- Simplify convex/convex.config.ts (remove betterAuth component)
- Delete convex/auth.ts, convex/http.ts, convex/userRoles.ts
- Remove userRoles table from convex/schema.ts
- Create src/services/clerk.ts with Clerk JS init, sign-in, sign-out, token, user metadata, UserButton
- Rewrite src/services/auth-state.ts backed by Clerk (same AuthUser/AuthSession interface)
- Delete src/services/auth-client.ts (better-auth client)
- Delete src/services/auth-token.ts (localStorage token scraping)
- Update .env.example with Clerk env vars, remove BETTER_AUTH_API_KEY

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(quick-4): UI components, runtime fetch, server-side JWT, CSP, and tests

- Delete AuthModal.ts, create AuthLauncher.ts (thin Clerk.openSignIn wrapper)
- Rewrite AuthHeaderWidget.ts to use Clerk UserButton + openSignIn
- Update event-handlers.ts to use AuthLauncher instead of AuthModal
- Rewrite runtime.ts enrichInitForPremium to use async getClerkToken()
- Rewrite server/auth-session.ts for jose-based JWT verification with cached JWKS
- Update vercel.json CSP: add *.clerk.accounts.dev to script-src and frame-src
- Add Clerk CSP tests to deploy-config.test.mjs
- Rewrite e2e/auth-ui.spec.ts for Clerk UI
- Rewrite auth-session.test.mts for jose-based validation
- Use dynamic import for @clerk/clerk-js to avoid Node.js test breakage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): allow Clerk Pro users to load premium data on web

The data-loader gated premium panel loading (stock-analysis, stock-backtest,
daily-market-brief) on WORLDMONITOR_API_KEY only, which is desktop-only.
Web users with Clerk Pro auth were seeing unlocked panels stuck on "Loading..."
because the requests were never made.

Added hasPremiumAccess() helper that checks for EITHER desktop API key OR
Clerk Pro role, matching the migration plan Phase 7 requirements.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): address PR #1812 review — all 4 merge blockers + 3 gaps

Blockers:
1. Remove stale Convex artifacts (http.js, userRoles.js, betterAuth
   component) from convex/_generated/api.d.ts
2. isProUser() now checks getAuthState().user?.role === 'pro' alongside
   legacy localStorage keys
3. Finance premium refresh scheduling now fires for Clerk Pro web users
   (not just API key holders)
4. JWT verification now validates audience: 'convex' to reject tokens
   scoped to other Clerk templates

Gaps:
5. auth-session tests: 10 new cases (valid pro/free, expired, wrong
   key/audience/issuer, missing sub/plan, JWKS reuse) using self-signed
   keys + local JWKS server
6. premium-stock-gateway tests: 4 new bearer token cases (pro→200,
   free→403, invalid→401, public unaffected)
7. docs/authentication.mdx rewritten for Clerk (removed all better-auth
   references, updated stack/files/env vars/roles sections)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address P1 reactive Pro UI + P2 daily-market-brief + P3 stale env vars

P1 — In-session Pro UI changes no longer require a full reload:
- setupExportPanel: removed early isProUser() return, always creates
  and relies on reactive subscribeAuthState show/hide
- setupPlaybackControl: same pattern — always creates, reactive gate
- Custom widget panels: always loaded regardless of Pro status
- Pro add-panel and MCP add-panel blocks: always rendered, shown/hidden
  reactively via subscribeAuthState callback
- Flight search wiring: always wired, checks Pro status inside callback
  so mid-session sign-ins work immediately

P2 — daily-market-brief added to hasPremiumAccess() block in loadAllData()
so Clerk Pro web users get initial data load (was only primed in
primeVisiblePanelData, missing from the general reload path)

P3 — Removed stale CONVEX_SITE_URL and VITE_CONVEX_SITE_URL from
docs/authentication.mdx env vars table (neither is referenced in codebase)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add isProUser import, populate PREMIUM_RPC_PATHS, and fix bearer token auth flow

- Added missing isProUser import in App.ts (fixes typecheck)
- Populated PREMIUM_RPC_PATHS with stock analysis endpoints
- Restructured gateway auth: trusted browser origins bypass API key for
  premium endpoints (client-side isProUser gate), while bearer token
  validation runs as a separate step for premium paths when present

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(gateway): require credentials for premium paths + defer free-tier enforcement until auth ready

P0: Removed trusted-origin bypass for premium endpoints — Origin header
is spoofable and cannot be a security boundary. Premium paths now always
require either an API key or valid bearer token.

P1: Deferred panel/source free-tier enforcement until auth state resolves.
Previously ran in the constructor before initAuthState(), causing Clerk Pro
users to have their panels/sources trimmed on every startup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): apply WorldMonitor design system to Clerk modal

Theme-aware appearance config passed to clerk.load(), openSignIn(),
and mountUserButton(). Dark mode: dark bg (#111), green primary
(#44ff88), monospace font. Light mode: white bg, green-600 primary
(#16a34a). Reads document.documentElement.dataset.theme at call time
so theme switches are respected.

* fix(auth): gate Clerk init and auth widget behind BETA_MODE

Clerk auth initialization and the Sign In header widget are now only
activated when localStorage `worldmonitor-beta-mode` is set to "true",
allowing silent deployment for internal testing before public rollout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(auth): gate Clerk init and auth widget behind isProUser()

Clerk auth initialization and the Sign In header widget are now only
activated when the user has wm-widget-key or wm-pro-key in localStorage
(i.e. isProUser() returns true), allowing silent deployment for internal
testing before public rollout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(data-loader): replace stale isProUser() with hasPremiumAccess()

loadMarketImplications() still referenced the removed isProUser import,
causing a TS2304 build error. Align with the rest of data-loader.ts
which uses hasPremiumAccess() (checks both API key and Clerk auth).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(auth): address PR #1812 review — P1 security fixes + P2 improvements

P1 fixes:
- Add algorithms: ['RS256'] allowlist to jwtVerify (prevents alg:none bypass)
- Reset loadPromise on Clerk init failure (allows retry instead of permanent breakage)

P2 fixes:
- Extract PREMIUM_RPC_PATHS to shared module (eliminates server/client divergence risk)
- Add fail-fast guard in convex/auth.config.ts for missing CLERK_JWT_ISSUER_DOMAIN
- Add 50s token cache with in-flight dedup to getClerkToken() (prevents concurrent races)
- Sync Clerk CSP entries to index.html and tauri.conf.json (previously only in vercel.json)
- Type clerkInstance as Clerk instead of any

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(auth): clear cached token on signOut()

Prevents stale token from being returned during the ≤50s cache window
after a user signs out.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Sebastien Melki <sebastien@anghami.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Sebastien Melki <sebastienmelki@gmail.com>
2026-03-26 13:47:22 +02:00
Elie Habib
aaf4c60b3e chore(deps): bump fast-xml-parser to 5.5.8 and Tauri to 2.10.3 (#1930) 2026-03-20 17:46:41 +04:00
Elie Habib
32ca22d69f feat(analytics): add Umami analytics via self-hosted instance (#1914)
* feat(analytics): add Umami analytics via self-hosted instance

Adds Umami analytics script from abacus.worldmonitor.app and updates
CSP headers in both index.html and vercel.json to allow the script.

* feat(analytics): complete Umami integration with event tracking

- Add data-domains to index.html script to exclude dev traffic
- Add Umami script to /pro page and blog (Base.astro)
- Add TypeScript Window.umami shim to vite-env.d.ts
- Wire analytics.ts facade to Umami (replaces PostHog no-ops):
  search, country clicks, map layers, panels, LLM usage, theme,
  language, variant switch, webcam, download, findings, deeplinks
- Add direct callsite tracking for: settings-open, mcp-connect-attempt,
  mcp-connect-success, mcp-panel-add, widget-ai-open/generate/success,
  news-summarize, news-sort-toggle, live-news-fullscreen,
  webcam-fullscreen, search-open (desktop/mobile/fab)

* fix(analytics): add Tauri CSP allowlist for Umami + skip programmatic layer events

- Add abacus.worldmonitor.app to Tauri CSP script-src and connect-src
  so Umami loads in the desktop WebView (analytics exception to the
  no-cloud-data rule — needed to know if desktop is used)
- Filter trackMapLayerToggle to user-initiated events only to avoid
  inflating counts with programmatic toggles on page load
2026-03-20 12:51:32 +04:00
Elie Habib
483d859ceb Triage security alerts (#1903)
* fix(cors): use ACAO: * for bootstrap to fix CF cache origin pinning

CF ignores Vary: Origin and pins the first request's ACAO header on the
cached response. Preview deployments from *.vercel.app got ACAO: worldmonitor.app
from CF's cache, blocking CORS. Bootstrap data is fully public (world events,
market prices, seismic data) so ACAO: * is safe and allows CF to cache one
entry valid for all origins. isDisallowedOrigin() still gates non-cache paths.

* chore: finish security triage

* fix(aviation): update isArray callback signature for fast-xml-parser 5.5.x

fast-xml-parser bumped from 5.4.2 to 5.5.7 changed the isArray callback's
second parameter type from string to unknown. Guard with typeof check before
calling .test() to satisfy the new type contract.

* docs: fix MD032 blank lines around lists in tradingview-screener-integration

* fix(security): address code review findings from PR #1903

- api/_json-response.js: add recursion depth limit (20) to sanitizeJsonValue
  and strip Error.cause chain alongside stack/stackTrace
- scripts/ais-relay.cjs: extract WORLD_BANK_COUNTRY_ALLOWLIST to module level
  to eliminate duplicate; clamp years param to [1,30] to prevent unbounded
  World Bank date ranges
- src-tauri/sidecar/local-api-server.mjs: use JSON.stringify for vq value
  in inline JS, consistent with safeVideoId/safeOrigin handling
- src/services/story-share.ts: simplify sanitizeStoryType to use typed array
  instead of repeated as-casts

* fix(desktop): use parent window origin for YouTube embed postMessage

Sidecar youtube-embed route was targeting the iframe's own localhost origin
for all window.parent.postMessage calls, so browsers dropped yt-ready/
yt-state/yt-error on Tauri builds where the parent is tauri://localhost or
asset://localhost. LiveNewsPanel and LiveWebcamsPanel already pass
parentOrigin=window.location.origin in the embed URL; the sidecar now reads,
validates, and uses it as the postMessage target for all player event
messages. The YT API playerVars origin/widget_referrer continue to use the
sidecar's own localhost origin which YouTube requires.

Also restore World Bank relay to a generic proxy: replace TECH_INDICATORS
membership check with a format-only regex so any valid indicator code
(NY.GDP.MKTP.CD etc.) is accepted, not just the 16 tech-sector codes.
2026-03-20 12:37:24 +04:00
Fayez Bast
8c6177b927 fix(cache): dedupe cache key handling and harden cache lifecycle safety (#1570)
* fix(cache): dedupe cache key handling and harden cache lifecycle safety

* fix(cache): address review findings in circuit-breaker hardening PR

- src/utils/index.ts: remove duplicate re-export of storage-quota symbols;
  single import at bottom of file now serves both local use (saveToStorage)
  and re-export to consumers
- src/services/market/index.ts: document first-wins semantics on
  uppercaseMetaMap so the intent is clear to future readers
- src/services/persistent-cache.ts: replace double-negation regex predicate
  with explicit suffix.length === 0 form for readability
- src-tauri/src/main.rs: replace !any(ch != ':') with is_empty() ||
  all(ch == ':') — same logic, no double negation
- tests/tech-readiness-circuit-breakers.test.mjs: note that the typescript
  import is the devDep already required by tsc

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-03-19 11:08:04 +04:00
Elie Habib
cf48144138 feat(widgets): add Exa web search + fix widget API endpoints (#1782)
* feat(widgets): add Exa web search + fix widget API endpoints

- Replace Tavily with Exa as primary stock-news search provider
  (Exa → Brave → SerpAPI → Google News RSS cascade)
- Add search_web tool to widget agent so AI can fetch live data
  about any topic beyond the pre-defined RPC catalog
- Exa primary (type:auto + content snippets), Brave fallback
- Fix all widget tool endpoints: /rpc/... paths were hitting
  Vercel catch-all and returning SPA HTML instead of JSON data
- Fix wm-widget-shell min-height causing fixed-size border that
  clipped AI widget content
- Add HTML response guard in tool handler
- Update env key: TAVILY_API_KEYS → EXA_API_KEYS throughout

* fix(stock-news): use type 'neural' for Exa search (type 'news' is invalid)
2026-03-17 19:25:08 +04:00
Elie Habib
2e36e57a75 fix(sidecar): block cloud fallback in Docker mode (#1726)
Self-hosted Docker instances must not proxy unhandled routes to
api.worldmonitor.app. When LOCAL_API_MODE=docker, cloudFallback is
forced to false regardless of LOCAL_API_CLOUD_FALLBACK env var.
Logs a warning if the user explicitly requested fallback.

Prevents self-hosted users from unknowingly sending traffic to the
production Vercel deployment.
2026-03-17 02:10:12 +04:00
Elie Habib
c6f5d6a8f1 fix(health): remove stale SEED_META for RPC-populated keys, bump to v2.6.5 (#1669)
riskScores and serviceStatuses have data but permanently stale seed-meta
(no longer written by cachedFetchJson after PR #1649). ON_DEMAND_KEYS
only affects EMPTY status, not STALE_SEED. Removing their SEED_META
entries so health doesn't check freshness for keys that can't update it.

Also bumps version to 2.6.5.
2026-03-15 19:27:18 +04:00
Elie Habib
442fb46a5d fix(sidecar): parallelize llm-health provider probes (#1564) 2026-03-14 12:59:03 +04:00
Jon Torrez
987ed03f5d feat(webcams): add webcam map layer with Windy API integration (#1540) (#1540)
- Webcam markers on flat, globe, and DeckGL maps with category-based icons
- Server-side spatial queries via Redis GEOSEARCH with quantized bbox caching
- Pinned webcams panel with localStorage persistence
- Seed script for Windy API with regional bounding boxes and adaptive splitting
- Input validation (webcamId regex + encodeURIComponent) and NaN projection guards
- Bandwidth optimizations: zoom threshold, bbox overlap check, 1s cooldown
- Client-side image cache with 200-entry FIFO eviction
- Globe altitude-based viewport estimation for webcam loading
- CSP updates for webcam iframe sources
- Seed-meta key for health.js freshness tracking
2026-03-14 09:34:54 +04:00
Jon Torrez
d9db5ab6c2 fix: move LLM health gate inside caching callbacks to prevent null-caching (#1522)
Moves isProviderAvailable() check from before cachedFetchJson() to inside
the fetcher callback. This ensures cache hits still serve valid data during
provider outages instead of returning empty results.

Changes:
- classify-event: health gate moved inside cachedFetchJson callback
- deduct-situation: same
- get-country-intel-brief: same
- summarize-article: same
- _batch-classify: break → return results on health gate failure
- callLlm (llm.ts): health gate added to provider chain
- local-api-server: /api/llm-health endpoint + startup warmup

Scope cleanup per review:
- Reverted LlmStatusIndicator (extracted to #1528)
- Reverted ACLED credential cleanup (extracted to #1530)
- Reverted isSidecar → isLocalDeployment rename (extracted to #1532)

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-03-14 08:25:12 +04:00
Elie Habib
15121f2092 chore: remove ACLED_EMAIL/ACLED_PASSWORD credential validation (#1530)
ACLED migrated to token-based auth (ACLED_ACCESS_TOKEN). The email/password
OAuth flow is no longer used. Remove the dead validation cases and drop
both keys from ALLOWED_ENV_KEYS.

Extracted from PR #1522 (scope split).

Co-authored-by: Jon Torrez <jrtorrez31337@users.noreply.github.com>
2026-03-13 20:26:03 +04:00
Nicolas Dos Santos
59cd313e16 fix(csp): add commodity variant to CSP and fix iframe variant navigation (#1506)
* fix(csp): add commodity variant to CSP and fix iframe variant navigation

- Add commodity.worldmonitor.app to frame-src and frame-ancestors in
  vercel.json and index.html CSP — was missing while all other variants
  were listed
- Open variant links in new tab when app runs inside an iframe to prevent
  sandbox navigation errors ("This content is blocked")
- Add allow-popups and allow-popups-to-escape-sandbox to pro page iframe
  sandbox attribute

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(csp): add missing variant subdomains to tauri.conf.json frame-src

Sync tauri.conf.json CSP with index.html and vercel.json by adding
finance, commodity, and happy worldmonitor.app subdomains to frame-src.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add PR screenshots for CSP fix

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-03-13 01:12:27 +04:00
Fayez Bast
36d0954720 feat(cache): key market quote breakers by symbol set (#1379)
* feat(cache): key market quote breakers by symbol set

* feat(cache): key market quote breakers by symbol set

* feat(cache): key market quote breakers by symbol set

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-03-13 00:21:08 +04:00
RepairYourTech
0420a54866 fix(acled): add OAuth token manager with automatic refresh (#1437)
* fix(acled): add OAuth token manager with automatic refresh

ACLED access tokens expire every 24 hours, but WorldMonitor stores a
static ACLED_ACCESS_TOKEN with no refresh logic — causing all ACLED
API calls to fail after the first day.

This commit adds `acled-auth.ts`, an OAuth token manager that:
- Exchanges ACLED_EMAIL + ACLED_PASSWORD for an access token (24h)
  and refresh token (14d) via the official ACLED OAuth endpoint
- Caches tokens in memory and auto-refreshes before expiry
- Falls back to static ACLED_ACCESS_TOKEN for backward compatibility
- Deduplicates concurrent refresh attempts
- Degrades gracefully when no credentials are configured

The only change to the existing `acled.ts` is replacing the synchronous
`process.env.ACLED_ACCESS_TOKEN` read with an async call to the new
`getAcledAccessToken()` helper.

Fixes #1283
Relates to #290

* fix: address review feedback on ACLED OAuth PR

- Use Redis (Upstash) as L2 token cache to survive Vercel Edge cold starts
  (in-memory cache retained as fast-path L1)
- Add CHROME_UA User-Agent header on OAuth token exchange and refresh
- Update seed script to use OAuth flow via getAcledToken() helper
  instead of raw process.env.ACLED_ACCESS_TOKEN
- Add security comment to .env.example about plaintext password trade-offs
- Sidecar ACLED_ACCESS_TOKEN case is a validation probe (tests user-provided
  value, not process.env) — data fetching delegates to handler modules

* feat(sidecar): add ACLED_EMAIL/ACLED_PASSWORD to env allowlist and validation

- Add ACLED_EMAIL and ACLED_PASSWORD to ALLOWED_ENV_KEYS set
- Add ACLED_EMAIL validation case (store-only, verified with password)
- Add ACLED_PASSWORD validation case with OAuth token exchange via
  acleddata.com/api/acled/user/login
- On successful login, store obtained OAuth token in ACLED_ACCESS_TOKEN
- Follows existing validation patterns (Cloudflare challenge handling,
  auth failure detection, User-Agent header)

* fix: address remaining review feedback (duplicate OAuth, em dashes, emoji)

- Extract shared ACLED OAuth helper into scripts/shared/acled-oauth.mjs
- Remove ~55 lines of duplicate OAuth logic from seed-unrest-events.mjs,
  now imports getAcledToken from the shared helper
- Replace em dashes with ASCII dashes in acled-auth.ts section comments
- Replace em dash with parentheses in sidecar validation message
- Remove emoji from .env.example security note

Addresses koala73's second review: MEDIUM (duplicate OAuth), LOW (em
dashes), LOW (emoji).

* fix: align sidecar OAuth endpoint, fix L1/L2 cache, cleanup artifacts

- Sidecar: switch from /api/acled/user/login (JSON) to /oauth/token
  (URL-encoded) to match server/_shared/acled-auth.ts exactly
- acled-auth.ts: check L2 Redis when L1 is expired, not only when L1
  is null (fixes stale L1 skipping fresher L2 from another isolate)
- acled-oauth.mjs: remove stray backslash on line 9
- seed-unrest-events.mjs: remove extra blank line at line 13

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
Co-authored-by: RepairYourTech <30200484+RepairYourTech@users.noreply.github.com>
2026-03-12 22:24:40 +04:00
Elie Habib
651cd3d08b feat(desktop): sidecar cloud proxy, domain handlers, and panel fixes (#1454)
* feat(desktop): compile domain handlers + add in-memory sidecar cache

The sidecar was broken for all 23 sebuf/RPC domain routes because
the build script (build-sidecar-handlers.mjs) never existed on main
while package.json already referenced it. This adds the missing script
and an in-memory TTL+LRU cache so the sidecar doesn't need Upstash Redis.

- Add scripts/build-sidecar-handlers.mjs (esbuild multi-entry, 23 domains)
- Add server/_shared/sidecar-cache.ts (500 entries, 50MB max, lazy sweep)
- Modify redis.ts getCachedJson/setCachedJson to use dynamic import for
  sidecar cache when LOCAL_API_MODE=tauri-sidecar (zero cost on Vercel Edge)
- Update tauri.conf.json beforeDevCommand to compile handlers
- Add gitignore pattern for compiled api/*/v1/[rpc].js

* fix(desktop): gate premium panel fetches and open footer links in browser

Skip oref-sirens and telegram-intel HTTP requests on desktop when
WORLDMONITOR_API_KEY is not present. Use absolute URLs for footer
links on desktop so the Tauri external link handler opens them in
the system browser instead of navigating within the webview.

* fix(desktop): cloud proxy, bootstrap timeouts, and panel data fixes

- Set Origin header on cloud proxy requests (fixes 401 from API key validator)
- Strip If-None-Match/If-Modified-Since headers (fixes stale 304 responses)
- Add cloud-preferred routing for market/economic/news/infrastructure/research
- Enable cloud fallback via LOCAL_API_CLOUD_FALLBACK env var in main.rs
- Increase bootstrap timeouts on desktop (8s/12s vs 3s/5s) for sidecar proxy hops
- Force per-feed RSS fallback on desktop (server digest has fewer categories)
- Add finance feeds to commodity variant (client + server)
- Remove desktop diagnostics from ServiceStatusPanel (show cloud statuses only)
- Restore DeductionPanel CSS from PR #1162
- Deduplicate repeated sidecar error logs
2026-03-12 06:50:30 +04:00
Elie Habib
f26c1b3016 chore: bump version to 2.6.1 with changelog (#1410)
Release 2.6.1 covering blog platform, country intelligence,
satellite imagery overhaul, and numerous fixes since 2.6.0.
2026-03-11 10:51:10 +04:00
Elie Habib
47c337014d revert: use youtube.com embeds and remove sandbox to fix bot-check (#1361)
Reverts commit 04af5ea8 which switched web webcam embeds back to
youtube-nocookie.com and restored sandbox. The nocookie domain triggers
YouTube's "Sign in to confirm you're not a bot" prompt, breaking all
live webcam feeds on the web app.

Changes:
- Web embeds: youtube-nocookie.com -> youtube.com (sends session cookies)
- Remove iframe sandbox attribute (allows storage-access to work)
- Add storage-access to iframe allow attribute
- Sidecar: restore autoplay-based MutationObserver gate
2026-03-10 07:39:55 +04:00
Jon Torrez
8bd4ab1cbf fix: resolve YouTube 'sign in to confirm' bot-check in embed panels (#1284)
* fix: resolve YouTube 'sign in to confirm' bot-check in embed panels

YouTube was showing a bot-verification prompt in the LiveWebcamsPanel
and LiveNewsPanel despite the user being logged into YouTube in the
same browser session.

LiveWebcamsPanel (primary fix):
- Changed embed domain from youtube-nocookie.com to youtube.com.
  The nocookie domain deliberately strips all cookies, so YouTube
  can never verify a signed-in session.
- Removed sandbox attribute which blocked the Storage Access API
  (allow-storage-access-by-user-activation was missing).
- Added storage-access to iframe allow attribute.

LiveNewsPanel:
- renderDesktopEmbed now passes origin and parentOrigin query params
  so postMessage is not silently dropped by the embed.
- Added storage-access to iframe allow attribute.
- Fixed MutationObserver target: was watching this.playerElement but
  YT.Player(domElement) replaces that div in its parent, so the
  observer never fired. Now observes playerContainer with a YouTube
  iframe filter, and YT.Player receives the element ID string so the
  iframe is inserted as a child of the div instead.

local-api-server.mjs (youtube-embed handler):
- MutationObserver patches inner YouTube iframe with storage-access.
- Added Permissions-Policy: storage-access=* response header.
- Embed page calls document.requestStorageAccess() on load.

api/youtube/embed.js (Vercel/edge path):
- Added tauri://localhost to ALLOWED_PARENT_ORIGINS.
- Added Permissions-Policy: storage-access=* response header.
- Embed page calls document.requestStorageAccess() on load.

* fix(pr-review): address review feedback on YouTube Storage Access API changes

- LiveWebcamsPanel: tested allow-storage-access-by-user-activation sandbox token
  as suggested; reverted — Chrome silently blocks Storage Access API even with
  the token present. Documented why sandbox removal is the only working approach.
- LiveWebcamsPanel: added comment documenting youtube-nocookie→youtube.com
  privacy trade-off as intentional.
- LiveNewsPanel: wrap YT.Player constructor in try/catch to disconnect
  storageObserver on error; add 10 s auto-disconnect timeout to prevent leaks.
- embed.js + local-api-server.mjs: scope permissions-policy storage-access to
  self + youtube.com rather than *.
- embed.js + local-api-server.mjs: add gesture-gated requestStorageAccess()
  fallback on first user interaction.
- embed.js: remove duplicate tauri://localhost from ALLOWED_PARENT_ORIGINS
  (already covered via ALLOWED_ORIGINS spread).

* fix(review): gate sidecar patch on storage-access, revert web webcam path

1. Sidecar MutationObserver: gate iframe patch on storage-access absence
   instead of autoplay absence. If YouTube ships iframes with autoplay
   already present, the old check would skip adding storage-access entirely.

2. Web webcam path: revert to youtube-nocookie.com and restore sandbox.
   The raw YouTube iframe cannot call requestStorageAccess() (no controlled
   bridge document), so switching to youtube.com only regressed privacy
   and sandbox security without actually fixing the bot-check.

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-03-10 07:00:07 +04:00
lspassos1
294984d315 fix(test): update stale origin-header assertion to match current behavior (#1329)
Since e14af08f (#709), the sidecar strips the browser Origin header but
immediately replaces it with `http://127.0.0.1:<port>` (line 1321 of
local-api-server.mjs). This ensures local handlers receive a valid
Origin for CORS while preventing browser-supplied origins from leaking
into server-to-server calls.

The test was written before that commit and still asserted
`originPresent: false`. Update the test to:
  - Assert `originPresent: true` (a localhost origin IS present)
  - Assert `originValue` equals `http://127.0.0.1:<port>` (verify it's
    the replaced localhost origin, not the browser's)
  - Rename the test to describe the actual behavior

npm run test:sidecar: 55/55 pass 
2026-03-09 23:57:49 +04:00
Elie Habib
595e3dbb86 feat: premium finance stock analysis suite (#1268)
* Add premium finance stock analysis suite

* docs: link premium finance from README

Add Premium Stock Analysis entry to the Finance & Markets section
with a link to docs/PREMIUM_FINANCE.md.

* fix: address review feedback on premium finance suite

- Chunk Redis pipelines into batches of 200 (Upstash limit)
- Add try-catch around cachedFetchJson in backtest handler
- Log warnings on Redis pipeline HTTP failures
- Include name in analyze-stock cache key to avoid collisions
- Change analyze-stock and backtest-stock gateway cache to 'slow'
- Add dedup guard for concurrent ledger generation
- Add SerpAPI date pre-filter (tbs=qdr:d/w)
- Extract sanitizeSymbol to shared module
- Extract buildEmptyAnalysisResponse helper
- Fix RSI to use Wilder's smoothing (matches TradingView)
- Add console.warn for daily brief summarization errors
- Fall back to stale data in loadStockBacktest on error
- Make daily-market-brief premium on all platforms
- Use word boundaries for short token headline matching
- Add stock-analysis 15-min refresh interval
- Stagger stock-analysis and backtest requests (200ms)
- Rename signalTone to stockSignalTone
2026-03-08 22:54:40 +04:00
Elie Habib
9129a3bbe3 chore: bump version to 2.6.0 (#1282)
* chore: bump version to 2.6.0

* fix: non-null assertion for SearchModal list access
2026-03-08 22:00:31 +04:00
Elie Habib
8d83aa02eb fix(economic): guard against undefined BIS and spending data (#1162)
* feat: premium panel gating, code cleanup, and backend simplifications

Recovered stranded changes from fix/desktop-premium-error-unification.

Premium gating:
- Add premium field ('locked'|'enhanced') to PanelConfig and LayerDefinition
- Panel.showLocked() with lock icon, CTA button, and _locked guard
- PRO badge for enhanced panels when no WM API key
- Exponential backoff auto-retry on showError() (15s→30s→60s→180s cap)
- Gate oref-sirens and telegram-intel panels behind WM API key
- Lock gpsJamming and iranAttacks layer toggles, badge ciiChoropleth
- Add tauri-titlebar drag region for custom titlebar

Code cleanup:
- Extract inline CSS from AirlineIntelPanel, WorldClockPanel to panels.css
- Remove unused showGeoError() from CountryBriefPage
- Remove dead geocodeFailed/retryBtn/closeBtn locale keys (20 files)
- Clean up var names and inline styles across 6 components

Backend:
- Remove seed-meta throttle from redis.ts (unnecessary complexity)
- Risk scores: call handler functions directly instead of raw Redis reads
- Update OpenRouter model to gpt-oss-safeguard-20b:nitro
- Add direct UCDP API fetching with version probing

Config:
- Remove titleBarStyle: Overlay from tauri.conf.json
- Add build:pro and build-sidecar-handlers to build:desktop
- Remove DXB/RUH from default aviation watchlist
- Simplify reverse-geocode (remove AbortController wrapper)

* fix: cast handler requests to any for API tsconfig compat

* fix: revert stale changes that conflict with merged PRs

Reverts files to main versions where old branch changes would
overwrite intentional fixes from PRs #1134, #1138, #1144, #1154:

- news/_shared.ts: keep gemini-2.5-flash model (not stale gpt-oss)
- redis.ts: keep seed-meta throttle from PR #1138
- reverse-geocode.ts: keep AbortController timeout from PR #1134
- CountryBriefPage.ts: keep showGeoError() from PR #1134
- country-intel.ts: keep showGeoError usage from PR #1134
- get-risk-scores.ts: revert non-existent imports
- watchlist.ts: keep DXB/RUH airports from PR #1144
- locales: restore geocodeFailed/retryBtn/closeBtn keys

* fix: neutralize language, parallel override loading, fetch timeout

- Rename conflict zone from "War" to "Border Conflict", intensity high→medium
- Rewrite description to factual language (no "open war" claim)
- Load country boundary overrides in parallel with main GeoJSON
- Neutralize comments/docs: reference Natural Earth source, remove political terms
- Add 60s timeout to Natural Earth fetch script (~24MB download)
- Add trailing newline to GeoJSON override file

* fix: restore caller messages in Panel errors and vessel expansion in popups

- Move UCDP direct-fetch cooldown after successful fetch to avoid
  suppressing all data for 10 minutes on a single failure
- Use caller-provided messages in showError/showRetrying instead of
  discarding them; respect autoRetrySeconds parameter
- Restore cluster-toggle click handler and expandable vessel list
  in military cluster popups
2026-03-07 09:43:27 +04:00
Elie Habib
739333aa80 docs: expand AGPL-3.0 license section in README (#1143)
* fix(desktop): settings UI redesign, IPC security hardening, release profile

Settings window:
- Add titlebar drag region (macOS traffic light clearance)
- Move Export/Import from Overview to Debug & Logs section
- Category cards grid changed to 3-column layout

Security (IPC trust boundary):
- Add require_trusted_window() to get_desktop_runtime_info, open_url,
  open_live_channels_window_command, open_youtube_login
- Validate base_url in open_live_channels_window_command (localhost-only http)

Performance:
- Add [profile.release] with fat LTO, codegen-units=1, strip, panic=abort
- Reuse reqwest::Client via app state with connection pooling
- Debounce window resize handler (150ms) in EventHandlerManager

* docs: expand AGPL-3.0 license section in README

Add plain-language explanation of AGPL-3.0 rights and obligations
including attribution requirements, copyleft conditions, network
use clause (SaaS must share source), and a use-case reference table.
2026-03-06 23:47:04 +04:00
Elie Habib
6ccda09246 fix(sidecar): upstream concurrency limiter, Yahoo rate gate, startup batching (#1145)
- Sidecar: add global concurrency limiter (max 6 concurrent upstream requests)
- Sidecar: add Yahoo Finance rate gate (600ms spacing) in fetch patch
- Sidecar: fix default remoteBase to api.worldmonitor.app
- data-loader: stagger startup tasks in batches of 4 with 300ms delay
- get-country-stock-index: add yahooGate() before Yahoo fetch
- tauri.conf: add titleBarStyle Overlay
2026-03-06 23:45:23 +04:00
Elie Habib
426994e343 fix(desktop): DRY debounce, error handling, retry cap (review follow-up) (#1084)
* fix(desktop): address code review findings — DRY debounce, error handling, retry cap

- Extract duplicated flush-scheduling into schedule_debounced_flush() helper
- Drop flush_scheduled lock before spawning thread to narrow lock scope
- Add .catch() to lazyPanel() for silent import failure visibility
- Convert happy-variant panels to use lazyPanel() helper (consistency + error handling)
- Cap flush retries at 5 to prevent infinite loop on persistent disk errors
- Only clear sidecar caches when at least one batch entry succeeded
- Log batch fallback error for debugging

* fix: remove unsafe type casts in happy-variant lazy panels

Move ctx property assignments into the loader callback where the
concrete type is known, eliminating all `as unknown as` double casts.
2026-03-06 01:57:50 +04:00
Elie Habib
e3afcd45b4 perf(desktop): debounce cache writes, batch secret push, lazy panels, pause hidden polls (#1077)
- Rust PersistentCache: generation-counter debounce (2s coalesce) + atomic
  flush via temp file + rename to prevent corruption on crash
- Sidecar: add /api/local-env-update-batch endpoint; loadDesktopSecrets()
  now pushes all secrets in 1 request instead of 23, with single-endpoint
  fallback for older sidecars
- App startup: waitForSidecarReady() polls service-status before bootstrap
  fetch so sidecar port-file races no longer cause silent fallback
- Lazy panel instantiation: 16 niche/variant panels converted to dynamic
  import().then() — disabled panels cost zero at cold boot
- pauseWhenHidden: true on RefreshScheduler, OREF alerts, and Gulf
  Economies poll loops — zero background network when app is hidden
2026-03-06 00:48:31 +04:00
Elie Habib
29ef8eae2f docs: update README with accurate counts and 9 new feature sections (#1071)
- Fix stale counts: 170+ feeds → 435+, 15 bootstrap keys → 38,
  28+ data sources → 31, 20+ search types → 24, panel counts
- Add Aviation Intelligence Panel documentation
- Add Customizable Market Watchlist section
- Add News Importance Scoring algorithm details
- Add Railway Seed Data Pipeline table (21 cron jobs)
- Add SmartPollLoop adaptive polling documentation
- Expand Prediction Markets with 4-tier fetch strategy
- Add Iran conflict monitoring layer details
- Add Mobile search sheet and FAB section
- Expand Regression Testing section (30 files, 554 tests)
- Expand Bootstrap Hydration with full 38-key tier listing
- Bump version 2.5.24 → 2.5.25
2026-03-05 23:40:37 +04:00
Elie Habib
fccfa79a29 fix: remove emrldco analytics and improve basemap fallback reliability (#1052)
- Remove emrldco.com analytics script and CSP entries from index.html,
  vercel.json, and tauri.conf.json
- Replace setStyle() basemap fallback with full map recreation — setStyle()
  after a failed initial style load leaves MapLibre in a broken state
- Add 403/Forbidden to error detection patterns for basemap failures
- Scope fallback to pre-style-load errors only (post-load tile errors
  don't warrant destroying a working map)
2026-03-05 12:39:22 +04:00
Elie Habib
e771c3c6e0 feat: add emrldco analytics script with lazy loading (#1000)
Loads the script on window.load event so it never blocks initial render.
CSP updated in both index.html and tauri.conf.json.
2026-03-04 20:33:11 +04:00
Dharun Ashokkumar
dc30db9ce9 fix: desktop youtube cloud fallback via sidecar hardcoded route (#917)
Fixes #903. YouTube live detection from the desktop sidecar now proxies directly to cloud via tryCloudFallback(), bypassing the cloudFallback flag (off by default). Matches the existing register-interest pattern.
2026-03-04 08:46:04 +04:00
Narvis Bot
5579e6bc5d fix: use 'cmd /c start' instead of 'explorer' to open URLs on Windows (#741)
explorer.exe treats URLs as file paths, opening a file dialog.
'cmd /c start' properly delegates URLs to the default browser.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 01:17:40 -08:00
Elie Habib
d1318781ff docs(readme): update stats, add 8 new sections, bump to v2.5.24 (#874)
Correct all stale numbers to match current codebase:
- Languages: 16 → 19 (added Czech, Greek, Korean)
- RSS feeds: 150+ → 170+, live channel pool: 30+ → 70+
- Airports: 128 → 107, AviationStack: 114 → 40
- Hotspots: 74 → 217, proto domains: 20 → 22
- Telegram: 27 → 26, OREF locations: 1,478 → 1,480
- Panel counts: 45/31/31/8 → 47/35/33/10

Add 8 new documentation sections:
- Bootstrap Hydration (2-tier parallel pre-fetch)
- Breaking News Alert Pipeline (5 origins)
- Cross-Stream Correlation Engine (14 signal types)
- Adaptive Refresh Scheduling (backoff, jitter, throttle)
- Localization Architecture (bundles, boost, RTL, fonts)
- Intelligence Analysis Tradecraft (SATs, ACH, gap awareness)
- Client-Side Circuit Breakers (IndexedDB persistence)
- Programmatic API Access (api.worldmonitor.app)

Expand Happy Monitor with humanity counters, conservation,
renewables, and giving detail. Add negative caching docs.
Bump version 2.5.23 → 2.5.24.
2026-03-03 08:36:49 +04:00
Nicolas Gomes Ferreira Dos Santos
ba95f62477 fix(sidecar): add required params to ACLED API key validation probe (#804)
* fix(sidecar): add required params to ACLED API key validation probe

The validation endpoint was calling ACLED without event_type, event_date,
or event_date_where parameters. The production code in acled.ts always
passes these — ACLED may reject requests missing them, causing valid
tokens to fail validation.

Add Protests event type and a 7-day date range to match production usage.

Fixes #290.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(military): harden USNI fleet report ship name regex

The ship extraction regex only matched <em> and <i> tags. If USNI
changes HTML to use <strong>, <b>, <span>, or plain text, all ship
parsing silently fails.

Broaden the regex to handle any inline HTML tag or no tag at all.
Add console.warn when a strike group section yields zero ships to
aid debugging when HTML format changes.

Addresses #197 (L-12).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-03 00:58:24 +04:00
Elie Habib
b423995363 feat(conflict): wire UCDP (#760)
* feat(conflict): wire UCDP API access token across full stack

UCDP API now requires an `x-ucdp-access-token` header. Renames the
stub `UC_DP_KEY` to `UCDP_ACCESS_TOKEN` (matching ACLED convention)
and wires it through Rust keychain, sidecar allowlist + verification,
handler fetch headers, feature toggles, and desktop settings UI.

- Rename UC_DP_KEY → UCDP_ACCESS_TOKEN in type system and labels
- Add ucdpConflicts feature toggle with required secret
- Add UCDP_ACCESS_TOKEN to Rust SUPPORTED_SECRET_KEYS (24→25)
- Add sidecar ALLOWED_ENV_KEYS entry + validation with dynamic GED version probing
- Handler sends x-ucdp-access-token header when token is present
- UC_DP_KEY fallback in handler for one-release migration window
- Update .env.example, desktop-readiness, and docs

* feat(conflict): pre-fetch UCDP events via Railway cron + Redis cache

Replace the 228-line edge handler that fetched UCDP GED API on every
request with a thin Redis reader. The heavy fetch logic (version
discovery, paginated backward fetch, 1-year trailing window filter)
now runs as a setInterval loop in the Railway relay (ais-relay.cjs)
every 6 hours, writing to Redis key conflict:ucdp-events:v1.

Changes:
- Add UCDP seed loop to ais-relay.cjs (6h interval, 6 pages, 2K cap)
- Rewrite list-ucdp-events.ts as thin Redis reader (35 lines)
- Add conflict:ucdp-events:v1 to bootstrap batch keys
- Protect key from cache-purge via durable data prefix
- Add manual-only seed-ucdp-events workflow + standalone script
- Rename panel "UCDP Events" → "Armed Conflict Events" in locale
- Add 24h TTL + 25h staleness check as safety nets
2026-03-02 16:17:17 +04:00
Nicolas Gomes Ferreira Dos Santos
d90c845621 fix(market): move Finnhub API key from query string to X-Finnhub-Token header (#744)
API keys in URL query strings can leak via server logs, proxy logs,
Referer headers, and error reporting tools. Finnhub supports both
authentication methods — this moves to the header-based approach.

Addresses #197 (L-16).

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
2026-03-02 12:37:38 +04:00
Elie Habib
8a414228b4 fix: harden windows installer update path and map resize behavior (#739) 2026-03-02 11:33:24 +04:00
Elie Habib
aa94b0fd5e fix(csp): allow localhost in media-src for proxied HLS & remove CNN HLS (#711)
CSP media-src only allowed https: — blocked <video> from loading HLS
streams through the sidecar proxy at http://127.0.0.1:PORT. Direct HLS
channels (Sky, DW, Fox) use https:// CDN URLs and worked; proxied
channels (CNBC, CNN) were silently blocked, falling back to YouTube.

Also remove CNN from PROXIED_HLS_MAP — the upstream stream is wrong.
2026-03-02 03:02:49 +04:00
Elie Habib
e14af08f2d fix(desktop): resolve sidecar 401s, variant lock, and registration form (#v2.5.23) (#709)
- Sidecar 401 fix: inject trusted localhost Origin on requests passed to
  handler modules. The handler's validateApiKey() was seeing empty Origin
  (stripped by toHeaders) + no API key → 401 for ALL desktop API calls.
- Variant fix: check localStorage FIRST when running in Tauri desktop,
  so .env.local VITE_VARIANT doesn't override user's variant selection.
- Registration: force-show form for email delivery testing.
- Bump version to 2.5.23.
2026-03-02 02:08:57 +04:00
Elie Habib
6adfda8061 chore: bump version to 2.5.22 & comprehensive README update (#706)
Bump version 2.5.21 → 2.5.22 across package.json, Cargo.toml, and
tauri.conf.json.

README: document 15+ recently shipped features that were missing from
the README — AI Deduction panel, Headline Memory (RAG), server-side
feed aggregation, Gulf Economies panel, TV Mode, mobile map with touch
gestures, fullscreen live video, 18+ HLS channels, breaking news
click-through, badge animation toggle, cache purge admin endpoint,
locale-aware feed boost, OREF Redis persistence + 1,478 Hebrew→English
translations, and Oceania region tab. Update PostHog → Vercel Analytics.
Add 21 new completed roadmap items.
2026-03-02 01:24:20 +04:00
Elie Habib
078a239ceb feat(live-news): add CNN & CNBC HLS streams via sidecar proxy (#682)
* feat(live-news): add CNN & CNBC HLS streams via sidecar proxy (desktop only)

Add /api/hls-proxy route to sidecar that proxies HLS manifests and
segments from allowlisted CDN hosts, injecting the required Referer
header that browsers cannot set. Rewrites m3u8 URLs so all segments
and encryption keys also route through the proxy.

Desktop gets native <video> HLS playback for CNN and CNBC; web falls
through to YouTube as before (no bandwidth cost on Vercel).

* fix(types): add missing @types/dompurify dev dependency
2026-03-01 21:06:18 +04:00
Elie Habib
36e36d8b57 Cost/traffic hardening, runtime fallback controls, and PostHog removal (#638)
- Remove PostHog analytics runtime and configuration
- Add API rate limiting (api/_rate-limit.js)
- Harden traffic controls across edge functions
- Add runtime fallback controls and data-loader improvements
- Add military base data scripts (fetch-mirta-bases, fetch-osm-bases)
- Gitignore large raw data files
- Settings playground prototypes
2026-03-01 11:53:20 +04:00
Elie Habib
cac2a4f5af fix(desktop): route register-interest to cloud when sidecar lacks CONVEX_URL (#639)
* fix(desktop): route register-interest to cloud when sidecar lacks CONVEX_URL

The waitlist registration endpoint needs Convex (cloud-only dependency).
The sidecar handler returned 503 without cloud fallback, and
getRemoteApiBaseUrl() returned '' on desktop (VITE_WS_API_URL unset),
so the settings window fetch resolved to tauri://localhost → 404.

Three-layer fix:
1. Sidecar: tryCloudFallback() when CONVEX_URL missing (proxies to
   https://worldmonitor.app via remoteBase)
2. runtime.ts: getRemoteApiBaseUrl() defaults to https://worldmonitor.app
   on desktop when VITE_WS_API_URL is unset
3. CI: add VITE_WS_API_URL=https://worldmonitor.app to all 4 desktop
   build steps

* chore(deps): bump posthog-js to fix pre-push typecheck
2026-03-01 11:46:31 +04:00
Elie Habib
d0a2a50506 fix(desktop): backoff on errors to stop CPU abuse + shrink settings window (#633)
Three bugs combine to burn 130% CPU when sidecar auth fails:

1. RefreshScheduler resets backoff multiplier to 1 (fastest) on error,
   causing failed endpoints to poll at base interval instead of backing off.
   Fix: exponential backoff on errors, same as unchanged-data path.

2. classify-event batch system ignores 401 (auth failure) — only pauses
   on 429/5xx. Hundreds of classify calls fire every 2s, each wasted.
   Fix: pause 120s on 401, matching the 429/5xx pattern.

3. Fetch patch retries every 401 (refresh token + retry), doubling all
   requests to the sidecar even when token refresh consistently fails.
   Fix: 60s cooldown after a retry-401 still returns 401.

Also shrinks settings window from 760→600px (min 620→480) to reduce
the empty whitespace below content on all tabs.
2026-03-01 10:53:54 +04:00
Fayez Bast
eb1d596c0a fix(linux): sanitize env for xdg-open in AppImage (#631) 2026-03-01 10:27:20 +04:00
Elie Habib
8a9aa2b254 fix(sidecar): add AVIATIONSTACK_API and ICAO_API_KEY to env allowlist (#632)
Both keys were added to Rust SUPPORTED_SECRET_KEYS and runtime-config.ts
but the sidecar's own ALLOWED_ENV_KEYS was never updated. This caused
"key not in allowlist" 403 when saving/verifying these keys from the
desktop settings UI.

Also adds AviationStack API validation in validateSecretAgainstProvider.
2026-03-01 10:23:37 +04:00
Elie Habib
5bb3696f7a chore: bump version to 2.5.21 (#605) 2026-03-01 02:40:41 +04:00