eliott/worldmonitor
1
0
Fork 0
mirror of https://github.com/koala73/worldmonitor.git synced 2026-04-25 17:14:57 +02:00
Code Issues Packages Projects Releases Wiki Activity
3,613 Commits 138 Branches 48 Tags
main
Commit Graph

165 Commits

Author SHA1 Message Date
Sebastien Melki
e68a7147dd chore(api): sebuf migration follow-ups (post-#3242) (#3287) 
* chore(api-manifest): rewrite brief-why-matters reason as proper internal-helper justification

Carried in from #3248 merge as a band-aid (called out in #3242 review followup
checklist item 7). The endpoint genuinely belongs in internal-helper —
RELAY_SHARED_SECRET-bearer auth, cron-only caller, never reached by dashboards
or partners. Same shape constraint as api/notify.ts.

Replaces the apologetic "filed here to keep the lint green" framing with a
proper structural justification: modeling it as a generated service would
publish internal cron plumbing as user-facing API surface.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* feat(lint): premium-fetch parity check for ServiceClients (closes #3279)

Adds scripts/enforce-premium-fetch.mjs — AST-walks src/, finds every
`new <ServiceClient>(...)` (variable decl OR `this.foo =` assignment),
tracks which methods each instance actually calls, and fails if any
called method targets a path in src/shared/premium-paths.ts
PREMIUM_RPC_PATHS without `{ fetch: premiumFetch }` on the constructor.

Per-call-site analysis (not class-level) keeps the trade/index.ts pattern
clean — publicClient with globalThis.fetch + premiumClient with
premiumFetch on the same TradeServiceClient class — since publicClient
never calls a premium method.

Wired into:
- npm run lint:premium-fetch
- .husky/pre-push (right after lint:rate-limit-policies)
- .github/workflows/lint-code.yml (right after lint:api-contract)

Found and fixed three latent instances of the HIGH(new) #1 class from
#3242 review (silent 401 → empty fallback for signed-in browser pros):

- src/services/correlation-engine/engine.ts — IntelligenceServiceClient
  built with no fetch option called deductSituation. LLM-assessment overlay
  on convergence cards never landed for browser pros without a WM key.
- src/services/economic/index.ts — EconomicServiceClient with
  globalThis.fetch called getNationalDebt. National-debt panel rendered
  empty for browser pros.
- src/services/sanctions-pressure.ts — SanctionsServiceClient with
  globalThis.fetch called listSanctionsPressure. Sanctions-pressure panel
  rendered empty for browser pros.

All three swap to premiumFetch (single shared client, mirrors the
supply-chain/index.ts justification — premiumFetch no-ops safely on
public methods, so the public methods on those clients keep working).

Verification:
- lint:premium-fetch clean (34 ServiceClient classes, 28 premium paths,
  466 src/ files analyzed)
- Negative test: revert any of the three to globalThis.fetch → exit 1
  with file:line and called-premium-method names
- typecheck + typecheck:api clean
- lint:api-contract / lint:rate-limit-policies / lint:boundaries clean
- tests/sanctions-pressure.test.mjs + premium-fetch.test.mts: 16/16 pass

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(military): fetchStaleFallback NEG_TTL=30s parity (closes #3277)

The legacy /api/military-flights handler had NEG_TTL = 30_000ms — a short
suppression window after a failed live + stale read so we don't Redis-hammer
the stale key during sustained relay+seed outages.

Carried into the sebuf list-military-flights handler:
- Module-scoped `staleNegUntil` timestamp (per-isolate on Vercel Edge,
  which is fine — each warm isolate gets its own 30s suppression window).
- Set whenever fetchStaleFallback returns null (key missing, parse fail,
  empty array after staleToProto filter, or thrown error).
- Checked at the entry of fetchStaleFallback before doing the Redis read.
- Test seam `_resetStaleNegativeCacheForTests()` exposed for unit tests.

Test pinned in tests/redis-caching.test.mjs: drives a stale-empty cycle
three times — first read hits Redis, second within window doesn't, after
test-only reset it does again.

Verified: 18/18 redis-caching tests pass, typecheck:api clean,
lint:premium-fetch clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lint): rate-limit-policies regex → import() (closes #3278)

The previous lint regex-parsed ENDPOINT_RATE_POLICIES from the source
file. That worked because the literal happens to fit a single line per
key today, but a future reformat (multi-line key wrap, formatter swap,
etc.) would silently break the lint without breaking the build —
exactly the failure mode that's worse than no lint at all.

Fix:
- Export ENDPOINT_RATE_POLICIES from server/_shared/rate-limit.ts.
- Convert scripts/enforce-rate-limit-policies.mjs to async + dynamic
  import() of the policy object directly. Same TS module that the
  gateway uses at runtime → no source-of-truth drift possible.
- Run via tsx (already a dev dep, used by test:data) so the .mjs
  shebang can resolve a .ts import.
- npm script swapped to `tsx scripts/...`. .husky/pre-push uses
  `npm run lint:rate-limit-policies` so no hook change needed.

Verified:
- Clean: 6 policies / 182 gateway routes.
- Negative test (rename a key to the original sanctions typo
  /api/sanctions/v1/lookup-entity): exit 1 with the same incident-
  attributed remedy message as before.
- Reformat test (split a single-line entry across multiple lines):
  still passes — the property is what's read, not the source layout.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(shipping/v2): alertThreshold: 0 preserved; drop dead validation branch (#3242 followup)

Before: alert_threshold was a plain int32. proto3 scalar default is 0, so
the handler couldn't distinguish "partner explicitly sent 0 (deliver every
disruption)" from "partner omitted the field (apply legacy default 50)" —
both arrived as 0 and got coerced to 50 by `> 0 ? : 50`. Silent intent-drop
for any partner who wanted every alert. The subsequent `alertThreshold < 0`
branch was also unreachable after that coercion.

After:
- Proto field is `optional int32 alert_threshold` — TS type becomes
  `alertThreshold?: number`, so omitted = undefined and explicit 0 stays 0.
- Handler uses `req.alertThreshold ?? 50` — undefined → 50, any number
  passes through unchanged.
- Dead `< 0 || > 100` runtime check removed; buf.validate `int32.gte = 0,
  int32.lte = 100` already enforces the range at the wire layer.

Partner wire contract: identical for the omit-field and 1..100 cases.
Only behavioural change is explicit 0 — previously impossible to request,
now honored per proto3 optional semantics.

Scoped `buf generate --path worldmonitor/shipping/v2` to avoid the full-
regen `@ts-nocheck` drift Seb documented in the #3242 PR comments.
Re-applied `@ts-nocheck` on the two regenerated files manually.

Tests:
- `alertThreshold 0 coerces to 50` flipped to `alertThreshold 0 preserved`.
- New test: `alertThreshold omitted (undefined) applies legacy default 50`.
- `rejects > 100` test removed — proto/wire validation handles it; direct
  handler calls intentionally bypass wire and the handler no longer carries
  a redundant runtime range check.

Verified: 18/18 shipping-v2-handler tests pass, typecheck + typecheck:api
clean, all 4 custom lints clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* docs(shipping/v2): document missing webhook delivery worker + DNS-rebinding contract (#3242 followup)

#3242 followup checklist item 6 from @koala73 — sanity-check that the
delivery worker honors the re-resolve-and-re-check contract that
isBlockedCallbackUrl explicitly delegates to it.

Audit finding: no delivery worker for shipping/v2 webhooks exists in this
repo. Grep across the entire tree (excluding generated/dist) shows the
only readers of webhook:sub:* records are the registration / inspection /
rotate-secret handlers themselves. No code reads them and POSTs to the
stored callbackUrl. The delivery worker is presumed to live in Railway
(separate repo) or hasn't been built yet — neither is auditable from
this repo.

Refreshes the comment block at the top of webhook-shared.ts to:
- explicitly state DNS rebinding is NOT mitigated at registration
- spell out the four-step contract the delivery worker MUST follow
  (re-validate URL, dns.lookup, re-check resolved IP against patterns,
   fetch with resolved IP + Host header preserved)
- flag the in-repo gap so anyone landing delivery code can't miss it

Tracking the gap as #3288 — acceptance there is "delivery worker imports
the patterns + helpers from webhook-shared.ts and applies the four steps
before each send." Action moves to wherever the delivery worker actually
lives (Railway likely).

No code change. Tests + lints unchanged.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(lint): add rate-limit-policies step (greptile P1 #3287)

Pre-push hook ran lint:rate-limit-policies but the CI workflow did not,
so fork PRs and --no-verify pushes bypassed the exact drift check the
lint was added to enforce (closes #3278). Adding it right after
lint:api-contract so it runs in the same context the lint was designed
for.

* refactor(lint): premium-fetch regex → import() + loop classRe (greptile P2 #3287)

Two fragilities greptile flagged on enforce-premium-fetch.mjs:

1. loadPremiumPaths regex-parsed src/shared/premium-paths.ts with
   /'(\/api\/[^']+)'/g — same class of silent drift we just removed
   from enforce-rate-limit-policies in #3278. Reformatting the source
   Set (double quotes, spread, helper-computed entries) would drop
   paths from the lint while leaving the runtime untouched. Fix: flip
   the shebang to `#!/usr/bin/env -S npx tsx` and dynamic-import
   PREMIUM_RPC_PATHS directly, mirroring the rate-limit pattern.
   package.json lint:premium-fetch now invokes via tsx too so the
   npm-script path matches direct execution.

2. loadClientClassMap ran classRe.exec once, silently dropping every
   ServiceClient after the first if a file ever contained more than
   one. Current codegen emits one class per file so this was latent,
   but a template change would ship un-linted classes. Fix: collect
   every class-open match with matchAll, slice each class body with
   the next class's start as the boundary, and scan methods per-body
   so method-to-class binding stays correct even with multiple
   classes per file.

Verification:
- lint:premium-fetch clean (34 classes / 28 premium paths / 466 files
  — identical counts to pre-refactor, so no coverage regression).
- Negative test: revert src/services/economic/index.ts to
  globalThis.fetch → exit 1 with file:line, bound var name, and
  premium method list (getNationalDebt). Restore → clean.
- lint:rate-limit-policies still clean.

* fix(shipping/v2): re-add alertThreshold handler range guard (greptile nit 1 #3287)

Wire-layer buf.validate enforces 0..100, but direct handler invocation
(internal jobs, test harnesses, future transports) bypasses it. Cheap
invariant-at-the-boundary — rejects < 0 or > 100 with ValidationError
before the record is stored.

Tests: restored the rejects-out-of-range cases that were dropped when the
branch was (correctly) deleted as dead code on the previous commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lint): premium-fetch method-regex → TS AST (greptile nits 2+5 #3287)

loadClientClassMap:
  The method regex `async (\w+)\s*\([^)]*\)\s*:\s*Promise<[^>]+>\s*\{\s*let
  path = "..."` assumed (a) no nested `)` in arg types, (b) no nested `>`
  in the return type, (c) `let path = "..."` as the literal first statement.
  Any codegen template shift would silently drop methods with the lint still
  passing clean — the same silent-drift class #3287 just closed on the
  premium-paths side.

  Now walks the service_client.ts AST, matches `export class *ServiceClient`,
  iterates `MethodDeclaration` members, and reads the first
  `let path: string = '...'` variable statement as a StringLiteral. Tolerant
  to any reformatting of arg/return types or method shape.

findCalls scope-blindness:
  Added limitation comment — the walker matches `<varName>.<method>()`
  anywhere in the file without respecting scope. Two constructions in
  different function scopes sharing a var name merge their called-method
  sets. No current src/ file hits this; the lint errs cautiously (flags
  both instances). Keeping the walker simple until scope-aware binding
  is needed.

webhook-shared.ts:
  Inlined issue reference (#3288) so the breadcrumb resolves without
  bouncing through an MDX that isn't in the diff.

Verification:
- lint:premium-fetch clean — 34 classes / 28 premium paths / 489 files.
  Pre-refactor: 34 / 28 / 466. Class + path counts identical; file bump
  is from the main-branch rebase, not the refactor.
- Negative test: revert src/services/economic/index.ts premiumFetch →
  globalThis.fetch. Lint exits 1 at `src/services/economic/index.ts:64:7`
  with `premium method(s) called: getNationalDebt`. Restore → clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(lint): rate-limit OpenAPI regex → yaml parser (greptile nit 3 #3287)

Input side (ENDPOINT_RATE_POLICIES) was flipped to live `import()` in
4e79d029. Output side (OpenAPI routes) still regex-scraped top-level
`paths:` keys with `/^\s{4}(\/api\/[^\s:]+):/gm` — hard-coded 4-space
indent. Any YAML formatter change (2-space indent, flow style, line
folding) would silently drop routes and let policy-drift slip through
— same silent-drift class the input-side fix closed.

Now uses the `yaml` package (already a dep) to parse each
.openapi.yaml and reads `doc.paths` directly.

Verification:
- Clean: 6 policies / 189 routes (was 182 — yaml parser picks up a
  handful the regex missed, closing a silent coverage gap).
- Negative test: rename policy key back to /api/sanctions/v1/lookup-entity
  → exits 1 with the same incident-attributed remedy. Restore → clean.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(codegen): regenerate unified OpenAPI bundle for alert_threshold proto change

The shipping/v2 webhook alert_threshold field was flipped from `int32` to
`optional int32` with an expanded doc comment in f3339464. That comment
now surfaces in the unified docs/api/worldmonitor.openapi.yaml bundle
(introduced by #3341). Regenerated with sebuf v0.11.1 to pick it up.

No behaviour change — bundle-only documentation drift.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-24 18:00:41 +03:00
Elie Habib
def94733a8 feat(agent-readiness): Agent Skills discovery index (#3310) (#3355) 
* feat(agent-readiness): Agent Skills discovery index (#3310)

Closes #3310. Ships the Agent Skills Discovery v0.2.0 manifest at
/.well-known/agent-skills/index.json plus two real, useful skills.

Skills are grounded in real sebuf proto RPCs:
- fetch-country-brief → GetCountryIntelBrief (public).
- fetch-resilience-score → GetResilienceScore (Pro / API key).

Each SKILL.md documents endpoint, auth, parameters, response shape,
worked curl, errors, and when not to use the skill.

scripts/build-agent-skills-index.mjs walks every
public/.well-known/agent-skills/<name>/SKILL.md, sha256s the bytes,
and emits index.json. Wired into prebuild + every variant build so a
deploy can never ship an index whose digests disagree with served files.

tests/agent-skills-index.test.mjs asserts the index is up-to-date
via the script's --check mode and recomputes every sha256 against
the on-disk SKILL.md bytes.

Discovery wiring:
- public/.well-known/api-catalog: new anchor entry with the
  agent-skills-index rel per RFC 9727 linkset shape.
- vercel.json: adds agent-skills-index rel to the homepage +
  /index.html Link headers; deploy-config required-rels list updated.

Canonical URLs use the apex (worldmonitor.app) since #3322 fixed
the apex redirect that previously hid .well-known paths.

* fix(agent-readiness): correct auth header + harden frontmatter parser (#3310)

Addresses review findings on #3310.

## P1 — auth header was wrong in both SKILL.md files

The published skills documented `Authorization: Bearer wm_live_...`,
but WorldMonitor API keys must be sent in `X-WorldMonitor-Key`.
`Authorization: Bearer` is for MCP/OAuth or Clerk JWTs — not raw
`wm_live_...` keys. Agents that followed the SKILL.md verbatim would
have gotten 401s despite holding valid keys.

fetch-country-brief also incorrectly claimed the endpoint was
"public"; server-to-server callers without a trusted browser origin
are rejected by `validateApiKey`, so agents do need a key there too.
Fixed both SKILL.md files to document `X-WorldMonitor-Key` and
cross-link docs/usage-auth as the canonical auth matrix.

## P2 — frontmatter parser brittleness

The hand-rolled parser used `indexOf('\n---', 4)` as the closing
fence, which matched any body line that happened to start with `---`.
Swapped for a regex that anchors the fence to its own line, and
delegated value parsing to js-yaml (already a project dep) so future
catalog growth (quoted colons, typed values, arrays) does not trip
new edge cases.

Added parser-contract tests that lock in the new semantics:
body `---` does not terminate the block, values with colons survive
intact, non-mapping frontmatter throws, and no-frontmatter files
return an empty mapping.

Index.json rebuilt against the updated SKILL.md bytes.
 2026-04-23 22:21:25 +04:00
Elie Habib
dff14ed344 feat(agent-readiness): RFC 9727 API catalog + native openapi.yaml serve (#3343) 
* feat(agent-readiness): RFC 9727 API catalog + native openapi.yaml serve

Closes #3309, part of epic #3306.

- Serves the sebuf-bundled OpenAPI spec natively at
  https://www.worldmonitor.app/openapi.yaml with correct
  application/yaml content-type (no Mintlify proxy hop).
  Build-time copy from docs/api/worldmonitor.openapi.yaml.
- Publishes RFC 9727 API catalog at /.well-known/api-catalog
  with service-desc pointing at the native URL, status rel
  pointing at /api/health, and a separate anchor for the MCP
  endpoint referencing its SEP-1649 card (#3311).

Refs PR #3341 (sebuf v0.11.1 bundle landed).

* test(deploy-config): update SPA catch-all regex assertion

The deploy-config guardrail hard-codes the SPA catch-all regex string
and asserts its Cache-Control is no-cache. The prior commit added
openapi.yaml to the exclusion list; this updates the test to match so
the guardrail continues to protect HTML entry caching.

* fix(agent-readiness): address Greptile review on PR #3343

- Extract openapi.yaml copy into named script `build:openapi` and
  prefix every web-variant build (build:full/tech/finance/happy/
  commodity). prebuild delegates to the same script so the default
  `npm run build` path is unchanged. Swap shell `cp` for Node's
  cpSync for cross-platform safety.
- Bump service-desc MIME type in /.well-known/api-catalog from
  application/yaml to application/vnd.oai.openapi (IANA-registered
  OpenAPI media type). Endpoint Content-Type stays application/yaml
  for browser/tool compatibility.

* fix(agent-readiness): P1 health href + guardrail tests on PR #3343

- status.href in /.well-known/api-catalog was pointing at
  https://api.worldmonitor.app/health (which serves the SPA HTML,
  not a health response). Corrected to /api/health, which returns
  the real {"status":"HEALTHY",...} JSON from api/health.js.
- Extend tests/deploy-config.test.mjs with assertions that would
  have caught this regression: linkset structure, status/service-
  desc href shapes, and presence of build:openapi across every
  web-variant build script.
 2026-04-23 18:46:35 +04:00
Elie Habib
a7bd1248ac chore(lint): exclude docs/brainstorms and docs/ideation from lint:md (#3332) 
Both dirs are in .gitignore (alongside docs/plans which was already
excluded from lint:md). Brings the lint:md exclusion list in sync with
.gitignore so pre-push hooks don't flag local-only working docs that
the repo isn't tracking anyway.

Observed failure mode before: writing a brainstorm doc locally under
docs/brainstorms/ for context during a session, then pushing any
feature branch → markdownlint runs against the untracked doc, blocks
the push until the local doc is deleted or its formatting is scrubbed
to markdownlint's satisfaction. The doc never enters git, so the lint
work is pure friction.

No functional change to shipping code.
 2026-04-23 11:25:14 +04:00
Elie Habib
58589144a5 fix(deps): promote yaml from transitive peer to top-level dependency (#3333) 
Railway seed-bundle-resilience-recovery crashed at 06:36:18 UTC on
first tick post-#3328 with:

  Error [ERR_MODULE_NOT_FOUND]: Cannot find package 'yaml' imported
  from /app/shared/swf-manifest-loader.mjs

`scripts/shared/swf-manifest-loader.mjs` (landed in #3319) imports
`parse` from `yaml` to read `docs/methodology/swf-classification-manifest.yaml`.
The package is present locally (as a peer of other deps), but Railway
installs production deps only — transitive peers don't land in
/app/node_modules, so the seeder exits 1 before any work.

Adding `yaml ^2.8.3` to `dependencies` so `npm ci` in the container
installs it. Version matches the already-on-disk resolution in
package-lock. No consumer changes needed.

Unblocks the first Sovereign-Wealth Railway run on the resilience-recovery bundle.
 2026-04-23 11:22:54 +04:00
Sebastien Melki
58e42aadf9 chore(api): enforce sebuf contract + migrate drifting endpoints (#3207) (#3242) 
* chore(api): enforce sebuf contract via exceptions manifest (#3207)

Adds api/api-route-exceptions.json as the single source of truth for
non-proto /api/ endpoints, with scripts/enforce-sebuf-api-contract.mjs
gating every PR via npm run lint:api-contract. Fixes the root-only blind
spot in the prior allowlist (tests/edge-functions.test.mjs), which only
scanned top-level *.js files and missed nested paths and .ts endpoints —
the gap that let api/supply-chain/v1/country-products.ts and friends
drift under proto domain URL prefixes unchallenged.

Checks both directions: every api/<domain>/v<N>/[rpc].ts must pair with
a generated service_server.ts (so a deleted proto fails CI), and every
generated service must have an HTTP gateway (no orphaned generated code).

Manifest entries require category + reason + owner, with removal_issue
mandatory for temporary categories (deferred, migration-pending) and
forbidden for permanent ones. .github/CODEOWNERS pins the manifest to
@SebastienMelki so new exceptions don't slip through review.

The manifest only shrinks: migration-pending entries (19 today) will be
removed as subsequent commits in this PR land each migration.

* refactor(maritime): migrate /api/ais-snapshot → maritime/v1.GetVesselSnapshot (#3207)

The proto VesselSnapshot was carrying density + disruptions but the frontend
also needed sequence, relay status, and candidate_reports to drive the
position-callback system. Those only lived on the raw relay passthrough, so
the client had to keep hitting /api/ais-snapshot whenever callbacks were
registered and fall back to the proto RPC only when the relay URL was gone.

This commit pushes all three missing fields through the proto contract and
collapses the dual-fetch-path into one proto client call.

Proto changes (proto/worldmonitor/maritime/v1/):
  - VesselSnapshot gains sequence, status, candidate_reports.
  - GetVesselSnapshotRequest gains include_candidates (query: include_candidates).

Handler (server/worldmonitor/maritime/v1/get-vessel-snapshot.ts):
  - Forwards include_candidates to ?candidates=... on the relay.
  - Separate 5-min in-memory caches for the candidates=on and candidates=off
    variants; they have very different payload sizes and should not share a slot.
  - Per-request in-flight dedup preserved per-variant.

Frontend (src/services/maritime/index.ts):
  - fetchSnapshotPayload now calls MaritimeServiceClient.getVesselSnapshot
    directly with includeCandidates threaded through. The raw-relay path,
    SNAPSHOT_PROXY_URL, DIRECT_RAILWAY_SNAPSHOT_URL and LOCAL_SNAPSHOT_FALLBACK
    are gone — production already routed via Vercel, the "direct" branch only
    ever fired on localhost, and the proto gateway covers both.
  - New toLegacyCandidateReport helper mirrors toDensityZone/toDisruptionEvent.

api/ais-snapshot.js deleted; manifest entry removed. Only reduced the codegen
scope to worldmonitor.maritime.v1 (buf generate --path) — regenerating the
full tree drops // @ts-nocheck from every client/server file and surfaces
pre-existing type errors across 30+ unrelated services, which is not in
scope for this PR.

Shape-diff vs legacy payload:
  - disruptions / density: proto carries the same fields, just with the
    GeoCoordinates wrapper and enum strings (remapped client-side via
    existing toDisruptionEvent / toDensityZone helpers).
  - sequence, status.{connected,vessels,messages}: now populated from the
    proto response — was hardcoded to 0/false in the prior proto fallback.
  - candidateReports: same shape; optional numeric fields come through as
    0 instead of undefined, which the legacy consumer already handled.

* refactor(sanctions): migrate /api/sanctions-entity-search → LookupSanctionEntity (#3207)

The proto docstring already claimed "OFAC + OpenSanctions" coverage but the
handler only fuzzy-matched a local OFAC Redis index — narrower than the
legacy /api/sanctions-entity-search, which proxied OpenSanctions live (the
source advertised in docs/api-proxies.mdx). Deleting the legacy without
expanding the handler would have been a silent coverage regression for
external consumers.

Handler changes (server/worldmonitor/sanctions/v1/lookup-entity.ts):
  - Primary path: live search against api.opensanctions.org/search/default
    with an 8s timeout and the same User-Agent the legacy edge fn used.
  - Fallback path: the existing OFAC local fuzzy match, kept intact for when
    OpenSanctions is unreachable / rate-limiting.
  - Response source field flips between 'opensanctions' (happy path) and
    'ofac' (fallback) so clients can tell which index answered.
  - Query validation tightened: rejects q > 200 chars (matches legacy cap).

Rate limiting:
  - Added /api/sanctions/v1/lookup-entity to ENDPOINT_RATE_POLICIES at 30/min
    per IP — matches the legacy createIpRateLimiter budget. The gateway
    already enforces per-endpoint policies via checkEndpointRateLimit.

Docs:
  - docs/api-proxies.mdx — dropped the /api/sanctions-entity-search row
    (plus the orphaned /api/ais-snapshot row left over from the previous
    commit in this PR).
  - docs/panels/sanctions-pressure.mdx — points at the new RPC URL and
    describes the OpenSanctions-primary / OFAC-fallback semantics.

api/sanctions-entity-search.js deleted; manifest entry removed.

* refactor(military): migrate /api/military-flights → ListMilitaryFlights (#3207)

Legacy /api/military-flights read a pre-baked Redis blob written by the
seed-military-flights cron and returned flights in a flat app-friendly
shape (lat/lon, lowercase enums, lastSeenMs). The proto RPC takes a bbox,
fetches OpenSky live, classifies server-side, and returns nested
GeoCoordinates + MILITARY_*_TYPE_* enum strings + lastSeenAt — same data,
different contract.

fetchFromRedis in src/services/military-flights.ts was doing nothing
sebuf-aware. Renamed it to fetchViaProto and rewrote to:

  - Instantiate MilitaryServiceClient against getRpcBaseUrl().
  - Iterate MILITARY_QUERY_REGIONS (PACIFIC + WESTERN) in parallel — same
    regions the desktop OpenSky path and the seed cron already use, so
    dashboard coverage tracks the analytic pipeline.
  - Dedup by hexCode across regions.
  - Map proto → app shape via new mapProtoFlight helper plus three reverse
    enum maps (AIRCRAFT_TYPE_REVERSE, OPERATOR_REVERSE, CONFIDENCE_REVERSE).

The seed cron (scripts/seed-military-flights.mjs) stays put: it feeds
regional-snapshot mobility, cross-source signals, correlation, and the
health freshness check (api/health.js: 'military:flights:v1'). None of
those read the legacy HTTP endpoint; they read the Redis key directly.
The proto handler uses its own per-bbox cache keys under the same prefix,
so dashboard traffic no longer races the seed cron's blob — the two paths
diverge by a small refresh lag, which is acceptable.

Docs: dropped the /api/military-flights row from docs/api-proxies.mdx.

api/military-flights.js deleted; manifest entry removed.

Shape-diff vs legacy:
  - f.location.{latitude,longitude} → f.lat, f.lon
  - f.aircraftType: MILITARY_AIRCRAFT_TYPE_TANKER → 'tanker' via reverse map
  - f.operator: MILITARY_OPERATOR_USAF → 'usaf' via reverse map
  - f.confidence: MILITARY_CONFIDENCE_LOW → 'low' via reverse map
  - f.lastSeenAt (number) → f.lastSeen (Date)
  - f.enrichment → f.enriched (with field renames)
  - Extra fields registration / aircraftModel / origin / destination /
    firstSeenAt now flow through where proto populates them.

* fix(supply-chain): thread includeCandidates through chokepoint status (#3207)

Caught by tsconfig.api.json typecheck in the pre-push hook (not covered
by the plain tsc --noEmit run that ran before I pushed the ais-snapshot
commit). The chokepoint status handler calls getVesselSnapshot internally
with a static no-auth request — now required to include the new
includeCandidates bool from the proto extension.

Passing false: server-internal callers don't need per-vessel reports.

* test(maritime): update getVesselSnapshot cache assertions (#3207)

The ais-snapshot migration replaced the single cachedSnapshot/cacheTimestamp
pair with a per-variant cache so candidates-on and candidates-off payloads
don't evict each other. Pre-push hook surfaced that tests/server-handlers
still asserted the old variable names. Rewriting the assertions to match
the new shape while preserving the invariants they actually guard:

  - Freshness check against slot TTL.
  - Cache read before relay call.
  - Per-slot in-flight dedup.
  - Stale-serve on relay failure (result ?? slot.snapshot).

* chore(proto): restore // @ts-nocheck on regenerated maritime files (#3207)

I ran 'buf generate --path worldmonitor/maritime/v1' to scope the proto
regen to the one service I was changing (to avoid the toolchain drift
that drops @ts-nocheck from 60+ unrelated files — separate issue). But
the repo convention is the 'make generate' target, which runs buf and
then sed-prepends '// @ts-nocheck' to every generated .ts file. My
scoped command skipped the sed step. The proto-check CI enforces the
sed output, so the two maritime files need the directive restored.

* refactor(enrichment): decomm /api/enrichment/{company,signals} legacy edge fns (#3207)

Both endpoints were already ported to IntelligenceService:
  - getCompanyEnrichment  (/api/intelligence/v1/get-company-enrichment)
  - listCompanySignals    (/api/intelligence/v1/list-company-signals)

No frontend callers of the legacy /api/enrichment/* paths exist. Removes:
  - api/enrichment/company.js, signals.js, _domain.js
  - api-route-exceptions.json migration-pending entries (58 remain)
  - docs/api-proxies.mdx rows for /api/enrichment/{company,signals}
  - docs/architecture.mdx reference updated to the IntelligenceService RPCs

Verified: typecheck, typecheck:api, lint:api-contract (89 files / 58 entries),
lint:boundaries, tests/edge-functions.test.mjs (136 pass),
tests/enrichment-caching.test.mjs (14 pass — still guards the intelligence/v1
handlers), make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(leads): migrate /api/{contact,register-interest} → LeadsService (#3207)

New leads/v1 sebuf service with two POST RPCs:
  - SubmitContact    → /api/leads/v1/submit-contact
  - RegisterInterest → /api/leads/v1/register-interest

Handler logic ported 1:1 from api/contact.js + api/register-interest.js:
  - Turnstile verification (desktop sources bypass, preserved)
  - Honeypot (website field) silently accepts without upstream calls
  - Free-email-domain gate on SubmitContact (422 ApiError)
  - validateEmail (disposable/offensive/typo-TLD/MX) on RegisterInterest
  - Convex writes via ConvexHttpClient (contactMessages:submit, registerInterest:register)
  - Resend notification + confirmation emails (HTML templates unchanged)

Shared helpers moved to server/_shared/:
  - turnstile.ts (getClientIp + verifyTurnstile)
  - email-validation.ts (disposable/offensive/MX checks)

Rate limits preserved via ENDPOINT_RATE_POLICIES:
  - submit-contact:    3/hour per IP (was in-memory 3/hr)
  - register-interest: 5/hour per IP (was in-memory 5/hr; desktop
    sources previously capped at 2/hr via shared in-memory map —
    now 5/hr like everyone else, accepting the small regression in
    exchange for Upstash-backed global limiting)

Callers updated:
  - pro-test/src/App.tsx contact form → new submit-contact path
  - src-tauri/sidecar/local-api-server.mjs cloud-fallback rewrites
    /api/register-interest → /api/leads/v1/register-interest when
    proxying; keeps local path for older desktop builds
  - src/services/runtime.ts isKeyFreeApiTarget allows both old and
    new paths through the WORLDMONITOR_API_KEY-optional gate

Tests:
  - tests/contact-handler.test.mjs rewritten to call submitContact
    handler directly; asserts on ValidationError / ApiError
  - tests/email-validation.test.mjs + tests/turnstile.test.mjs
    point at the new server/_shared/ modules

Deleted: api/contact.js, api/register-interest.js, api/_ip-rate-limit.js,
api/_turnstile.js, api/_email-validation.js, api/_turnstile.test.mjs.
Manifest entries removed (58 → 56). Docs updated (api-platform,
api-commerce, usage-rate-limits).

Verified: npm run typecheck + typecheck:api + lint:api-contract
(88 files / 56 entries) + lint:boundaries pass; full test:data
(5852 tests) passes; make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(pro-test): rebuild bundle for leads/v1 contact form (#3207)

Updates the enterprise contact form to POST to /api/leads/v1/submit-contact
(old path /api/contact removed in the previous commit).

Bundle is rebuilt from pro-test/src/App.tsx source change in 9ccd309d.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): address HIGH review findings 1-3 (#3207)

Three review findings from @koala73 on the sebuf-migration PR, all
silent bugs that would have shipped to prod:

### 1. Sanctions rate-limit policy was dead code

ENDPOINT_RATE_POLICIES keyed the 30/min budget under
/api/sanctions/v1/lookup-entity, but the generated route (from the
proto RPC LookupSanctionEntity) is /api/sanctions/v1/lookup-sanction-entity.
hasEndpointRatePolicy / getEndpointRatelimit are exact-string pathname
lookups, so the mismatch meant the endpoint fell through to the
generic 600/min global limiter instead of the advertised 30/min.

Net effect: the live OpenSanctions proxy endpoint (unauthenticated,
external upstream) had 20x the intended rate budget. Fixed by renaming
the policy key to match the generated route.

### 2. Lost stale-seed fallback on military-flights

Legacy api/military-flights.js cascaded military:flights:v1 →
military:flights:stale:v1 before returning empty. The new proto
handler went straight to live OpenSky/relay and returned null on miss.

Relay or OpenSky hiccup used to serve stale seeded data (24h TTL);
under the new handler it showed an empty map. Both keys are still
written by scripts/seed-military-flights.mjs on every run — fix just
reads the stale key when the live fetch returns null, converts the
seed's app-shape flights (flat lat/lon, lowercase enums, lastSeenMs)
to the proto shape (nested GeoCoordinates, enum strings, lastSeenAt),
and filters to the request bbox.

Read via getRawJson (unprefixed) to match the seed cron's writes,
which bypass the env-prefix system.

### 3. Hex-code casing mismatch broke getFlightByHex

The seed cron writes hexCode: icao24.toUpperCase() (uppercase);
src/services/military-flights.ts:getFlightByHex uppercases the lookup
input: f.hexCode === hexCode.toUpperCase(). The new proto handler
preserved OpenSky's lowercase icao24, and mapProtoFlight is a
pass-through. getFlightByHex was silently returning undefined for
every call after the migration.

Fix: uppercase in the proto handler (live + stale paths), and document
the invariant in a comment on MilitaryFlight.hex_code in
military_flight.proto so future handlers don't re-break it.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries) / lint:boundaries clean
- tests/edge-functions.test.mjs 130 pass
- make generate zero-diff (openapi spec regenerated for proto comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): restore desktop 2/hr rate cap on register-interest (#3207)

Addresses HIGH review finding #4 from @koala73. The legacy
api/register-interest.js applied a nested 2/hr per-IP cap when
`source === 'desktop-settings'`, on top of the generic 5/hr endpoint
budget. The sebuf migration lost this — desktop-source requests now
enjoy the full 5/hr cap.

Since `source` is an unsigned client-supplied field, anyone sending
`source: 'desktop-settings'` skips Turnstile AND gets 5/hr. Without
the tighter cap the Turnstile bypass is cheaper to abuse.

Added `checkScopedRateLimit` to `server/_shared/rate-limit.ts` — a
reusable second-stage Upstash limiter keyed on an opaque scope string
+ caller identifier. Fail-open on Redis errors to match existing
checkRateLimit / checkEndpointRateLimit semantics. Handlers that need
per-subscope caps on top of the gateway-level endpoint budget use this
helper.

In register-interest: when `isDesktopSource`, call checkScopedRateLimit
with scope `/api/leads/v1/register-interest#desktop`, limit=2, window=1h,
IP as identifier. On exceeded → throw ApiError(429).

### What this does not fix

This caps the blast radius of the Turnstile bypass but does not close
it — an attacker sending `source: 'desktop-settings'` still skips
Turnstile (just at 2/hr instead of 5/hr). The proper fix is a signed
desktop-secret header that authenticates the bypass; filed as
follow-up #3252. That requires coordinated Tauri build + Vercel env
changes out of scope for #3207.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries)
- tests/edge-functions.test.mjs + contact-handler.test.mjs (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): MEDIUM + LOW + rate-limit-policy CI check (#3207)

Closes out the remaining @koala73 review findings from #3242 that
didn't already land in the HIGH-fix commits, plus the requested CI
check that would have caught HIGH #1 (dead-code policy key) at
review time.

### MEDIUM #5 — Turnstile missing-secret policy default

Flip `verifyTurnstile`'s default `missingSecretPolicy` from `'allow'`
to `'allow-in-development'`. Dev with no secret = pass (expected
local); prod with no secret = reject + log. submit-contact was
already explicitly overriding to `'allow-in-development'`;
register-interest was silently getting `'allow'`. Safe default now
means a future missing-secret misconfiguration in prod gets caught
instead of silently letting bots through. Removed the now-redundant
override in submit-contact.

### MEDIUM #6 — Silent enum fallbacks in maritime client

`toDisruptionEvent` mapped `AIS_DISRUPTION_TYPE_UNSPECIFIED` / unknown
enum values → `gap_spike` / `low` silently. Refactored to return null
when either enum is unknown; caller filters nulls out of the array.
Handler doesn't produce UNSPECIFIED today, but the `gap_spike`
default would have mislabeled the first new enum value the proto
ever adds — dropping unknowns is safer than shipping wrong labels.

### LOW — Copy drift in register-interest email

Email template hardcoded `435+ Sources`; PR #3241 bumped marketing to
`500+`. Bumped in the rewritten file to stay consistent.

The `as any` on Convex mutation names carried over from legacy and
filed as follow-up #3253.

### Rate-limit-policy coverage lint

`scripts/enforce-rate-limit-policies.mjs` validates every key in
`ENDPOINT_RATE_POLICIES` resolves to a proto-generated gateway route
by cross-referencing `docs/api/*.openapi.yaml`. Fails with the
sanctions-entity-search incident referenced in the error message so
future drift has a paper trail.

Wired into package.json (`lint:rate-limit-policies`) and the pre-push
hook alongside `lint:boundaries`. Smoke-tested both directions —
clean repo passes (5 policies / 175 routes), seeded drift (the exact
HIGH #1 typo) fails with the advertised remedy text.

### Verified
- `lint:rate-limit-policies` ✓
- `typecheck` + `typecheck:api` ✓
- `lint:api-contract` ✓ (56 entries)
- `lint:boundaries` ✓
- edge-functions + contact-handler tests (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 5): decomm /api/eia/* + migrate /api/satellites → IntelligenceService (#3207)

Both targets turned out to be decomm-not-migration cases. The original
plan called for two new services (economic/v1.GetEiaSeries +
natural/v1.ListSatellitePositions) but research found neither was
needed:

### /api/eia/[[...path]].js — pure decomm, zero consumers

The "catch-all" is a misnomer — only two paths actually worked,
/api/eia/health and /api/eia/petroleum, both Redis-only readers.
Zero frontend callers in src/. Zero server-side readers. Nothing
consumes the `energy:eia-petroleum:v1` key that seed-eia-petroleum.mjs
writes daily.

The EIA data the frontend actually uses goes through existing typed
RPCs in economic/v1: GetEnergyPrices, GetCrudeInventories,
GetNatGasStorage, GetEnergyCapacity. None of those touch /api/eia/*.

Building GetEiaSeries would have been dead code. Deleted the legacy
file + its test (tests/api-eia-petroleum.test.mjs — it only covered
the legacy endpoint, no behavior to preserve). Empty api/eia/ dir
removed.

**Note for review:** the Redis seed cron keeps running daily and
nothing consumes it. If that stays unused, seed-eia-petroleum.mjs
should be retired too (separate PR). Out of scope for sebuf-migration.

### /api/satellites.js — Learning #2 strikes again

IntelligenceService.ListSatellites already exists at
/api/intelligence/v1/list-satellites, reads the same Redis key
(intelligence:satellites:tle:v1), and supports an optional country
filter the legacy didn't have.

One frontend caller in src/services/satellites.ts needed to switch
from `fetch(toApiUrl('/api/satellites'))` to the typed
IntelligenceServiceClient.listSatellites. Shape diff was tiny —
legacy `noradId` became proto `id` (handler line 36 already picks
either), everything else identical. alt/velocity/inclination in the
proto are ignored by the caller since it propagates positions
client-side via satellite.js.

Kept the client-side cache + failure cooldown + 20s timeout (still
valid concerns at the caller level).

### Manifest + docs
- api-route-exceptions.json: 56 → 54 entries (both removed)
- docs/api-proxies.mdx: dropped the two rows from the Raw-data
  passthroughs table

### Verified
- typecheck + typecheck:api ✓
- lint:api-contract (54 entries) / lint:boundaries / lint:rate-limit-policies ✓
- tests/edge-functions.test.mjs 127 pass (down from 130 — 3 tests were
  for the deleted eia endpoint)
- make generate zero-diff (no proto changes)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 6): migrate /api/supply-chain/v1/{country-products,multi-sector-cost-shock} → SupplyChainService (#3207)

Both endpoints were hand-rolled TS handlers sitting under a proto URL prefix —
the exact drift the manifest guardrail flagged. Promoted both to typed RPCs:

- GetCountryProducts → /api/supply-chain/v1/get-country-products
- GetMultiSectorCostShock → /api/supply-chain/v1/get-multi-sector-cost-shock

Handlers preserve the existing semantics: PRO-gate via isCallerPremium(ctx.request),
iso2 / chokepointId validation, raw bilateral-hs4 Redis read (skip env-prefix to
match seeder writes), CHOKEPOINT_STATUS_KEY for war-risk tier, and the math from
_multi-sector-shock.ts unchanged. Empty-data and non-PRO paths return the typed
empty payload (no 403 — the sebuf gateway pattern is empty-payload-on-deny).

Client wrapper switches from premiumFetch to client.getCountryProducts/
client.getMultiSectorCostShock. Legacy MultiSectorShock / MultiSectorShockResponse /
CountryProductsResponse names remain as type aliases of the generated proto types
so CountryBriefPanel + CountryDeepDivePanel callsites compile with zero churn.

Manifest 54 → 52. Rate-limit gateway routes 175 → 177.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(gateway): add cache-tier entries for new supply-chain RPCs (#3207)

Pre-push tests/route-cache-tier.test.mjs caught the missing entries.
Both PRO-gated, request-varying — match the existing supply-chain PRO cohort
(get-country-cost-shock, get-bypass-options, etc.) at slow-browser tier.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 7): migrate /api/scenario/v1/{run,status,templates} → ScenarioService (#3207)

Promote the three literal-filename scenario endpoints to a typed sebuf
service with three RPCs:

  POST /api/scenario/v1/run-scenario        (RunScenario)
  GET  /api/scenario/v1/get-scenario-status (GetScenarioStatus)
  GET  /api/scenario/v1/list-scenario-templates (ListScenarioTemplates)

Preserves all security invariants from the legacy handlers:
- 405 for wrong method (sebuf service-config method gate)
- scenarioId validation against SCENARIO_TEMPLATES registry
- iso2 regex ^[A-Z]{2}$
- JOB_ID_RE path-traversal guard on status
- Per-IP 10/min rate limit (moved to gateway ENDPOINT_RATE_POLICIES)
- Queue-depth backpressure (>100 → 429)
- PRO gating via isCallerPremium
- AbortSignal.timeout on every Redis pipeline (runRedisPipeline helper)

Wire-level diffs vs legacy:
- Per-user RL now enforced at the gateway (same 10/min/IP budget).
- Rate-limit response omits Retry-After header; retryAfter is in the
  body per error-mapper.ts convention.
- ListScenarioTemplates emits affectedHs2: [] when the registry entry
  is null (all-sectors sentinel); proto repeated cannot carry null.
- RunScenario returns { jobId, status } (no statusUrl field — unused
  by SupplyChainPanel, drop from wire).

Gateway wiring:
- server/gateway.ts RPC_CACHE_TIER: list-scenario-templates → 'daily'
  (matches legacy max-age=3600); get-scenario-status → 'slow-browser'
  (premium short-circuit target, explicit entry required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: swap old run/status for the new
  run-scenario/get-scenario-status paths.
- api/scenario/v1/{run,status,templates}.ts deleted; 3 manifest
  exceptions removed (63 → 52 → 49 migration-pending).

Client:
- src/services/scenario/index.ts — typed client wrapper using
  premiumFetch (injects Clerk bearer / API key).
- src/components/SupplyChainPanel.ts — polling loop swapped from
  premiumFetch strings to runScenario/getScenarioStatus. Hard 20s
  timeout on run preserved via AbortSignal.any.

Tests:
- tests/scenario-handler.test.mjs — 18 new handler-level tests
  covering every security invariant + the worker envelope coercion.
- tests/edge-functions.test.mjs — scenario sections removed,
  replaced with a breadcrumb pointer to the new test file.

Docs: api-scenarios.mdx, scenario-engine.mdx, usage-rate-limits.mdx,
usage-errors.mdx, supply-chain.mdx refreshed with new paths.

Verified: typecheck, typecheck:api, lint:api-contract (49 entries),
lint:rate-limit-policies (6/180), lint:boundaries, route-cache-tier
(parity), full edge-functions (117) + scenario-handler (18).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 8): migrate /api/v2/shipping/{route-intelligence,webhooks} → ShippingV2Service (#3207)

Partner-facing endpoints promoted to a typed sebuf service. Wire shape
preserved byte-for-byte (camelCase field names, ISO-8601 fetchedAt, the
same subscriberId/secret formats, the same SET + SADD + EXPIRE 30-day
Redis pipeline). Partner URLs /api/v2/shipping/* are unchanged.

RPCs landed:
- GET  /route-intelligence  → RouteIntelligence  (PRO, slow-browser)
- POST /webhooks            → RegisterWebhook    (PRO)
- GET  /webhooks            → ListWebhooks       (PRO, slow-browser)

The existing path-parameter URLs remain on the legacy edge-function
layout because sebuf's HTTP annotations don't currently model path
params (grep proto/**/*.proto for `path: "{…}"` returns zero). Those
endpoints are split into two Vercel dynamic-route files under
api/v2/shipping/webhooks/, behaviorally identical to the previous
hybrid file but cleanly separated:
- GET  /webhooks/{subscriberId}                → [subscriberId].ts
- POST /webhooks/{subscriberId}/rotate-secret  → [subscriberId]/[action].ts
- POST /webhooks/{subscriberId}/reactivate     → [subscriberId]/[action].ts

Both get manifest entries under `migration-pending` pointing at #3207.

Other changes
- scripts/enforce-sebuf-api-contract.mjs: extended GATEWAY_RE to accept
  api/v{N}/{domain}/[rpc].ts (version-first) alongside the canonical
  api/{domain}/v{N}/[rpc].ts; first-use of the reversed ordering is
  shipping/v2 because that's the partner contract.
- vite.config.ts: dev-server sebuf interceptor regex extended to match
  both layouts; shipping/v2 import + allRoutes entry added.
- server/gateway.ts: RPC_CACHE_TIER entries for /api/v2/shipping/
  route-intelligence + /webhooks (slow-browser; premium-gated endpoints
  short-circuit to slow-browser but the entries are required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: route-intelligence + webhooks added.
- tests/shipping-v2-handler.test.mjs: 18 handler-level tests covering
  PRO gate, iso2/cargoType/hs2 coercion, SSRF guards (http://, RFC1918,
  cloud metadata, IMDS), chokepoint whitelist, alertThreshold range,
  secret/subscriberId format, pipeline shape + 30-day TTL, cross-tenant
  owner isolation, `secret` omission from list response.

Manifest delta
- Removed: api/v2/shipping/route-intelligence.ts, api/v2/shipping/webhooks.ts
- Added:   api/v2/shipping/webhooks/[subscriberId].ts (migration-pending)
- Added:   api/v2/shipping/webhooks/[subscriberId]/[action].ts (migration-pending)
- Added:   api/internal/brief-why-matters.ts (internal-helper) — regression
  surface from the #3248 main merge, which introduced the file without a
  manifest entry. Filed here to keep the lint green; not strictly in scope
  for commit 8 but unblocking.

Net result: 49 → 47 `migration-pending` entries (one net-removal even
though webhook path-params stay pending, because two files collapsed
into two dynamic routes).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 1): SupplyChainServiceClient must use premiumFetch (#3207)

Signed-in browser pro users were silently hitting 401 on 8 supply-chain
premium endpoints (country-products, multi-sector-cost-shock,
country-chokepoint-index, bypass-options, country-cost-shock,
sector-dependency, route-explorer-lane, route-impact). The shared
client was constructed with globalThis.fetch, so no Clerk bearer or
X-WorldMonitor-Key was injected. The gateway's validateApiKey runs
with forceKey=true for PREMIUM_RPC_PATHS and 401s before isCallerPremium
is consulted. The generated client's try/catch collapses the 401 into
an empty-fallback return, leaving panels blank with no visible error.

Fix is one line at the client constructor: swap globalThis.fetch for
premiumFetch. The same pattern is already in use for insider-transactions,
stock-analysis, stock-backtest, scenario, trade (premiumClient) — this
was an omission on this client, not a new pattern.

premiumFetch no-ops safely when no credentials are available, so the
5 non-premium methods on this client (shippingRates, chokepointStatus,
chokepointHistory, criticalMinerals, shippingStress) continue to work
unchanged.

This also fixes two panels that were pre-existing latently broken on
main (chokepoint-index, bypass-options, etc. — predating #3207, not
regressions from it). Commit 6 expanded the surface by routing two more
methods through the same buggy client; this commit fixes the class.

From koala73 review (#3242 second-pass, HIGH new #1):
> Exact class PR #3233 fixed for RegionalIntelligenceBoard /
> DeductionPanel / trade / country-intel. Supply-chain was not in
> #3233's scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 2): restore 400 on input-shape errors for 2 supply-chain handlers (#3207)

Commit 6 collapsed all non-happy paths into empty-200 on
`get-country-products` and `get-multi-sector-cost-shock`, including
caller-bug cases that legacy returned 400 for:

- get-country-products: malformed iso2 → empty 200 (was 400)
- get-multi-sector-cost-shock: malformed iso2 / missing chokepointId /
  unknown chokepointId → empty 200 (was 400)

The commit message for 6 called out the 403-for-non-pro → empty-200
shift ("sebuf gateway pattern is empty-payload-on-deny") but not the
400 shift. They're different classes:

- Empty-payload-200 for PRO-deny: intentional contract change, already
  documented and applied across the service. Generated clients treat
  "you lack PRO" as "no data" — fine.
- Empty-payload-200 for malformed input: caller bug silently masked.
  External API consumers can't distinguish "bad wiring" from "genuinely
  no data", test harnesses lose the signal, bad calling code doesn't
  surface in Sentry.

Fix: `throw new ValidationError(violations)` on the 3 input-shape
branches. The generated sebuf server maps ValidationError → HTTP 400
(see src/generated/server/.../service_server.ts and leads/v1 which
already uses this pattern).

PRO-gate deny stays as empty-200 — that contract shift was intentional
and is preserved.

Regression tests added at tests/supply-chain-validation.test.mjs (8
cases) pinning the three-way contract:
- bad input                         → 400 (ValidationError)
- PRO-gate deny on valid input      → 200 empty
- valid PRO input, no data in Redis → 200 empty (unchanged)

From koala73 review (#3242 second-pass, HIGH new #2).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 3): restore statusUrl on RunScenarioResponse + document 202→200 wire break (#3207)

Commit 7 silently shifted /api/scenario/v1/run-scenario's response
contract in two ways that the commit message covered only partially:

1. HTTP 202 Accepted → HTTP 200 OK
2. Dropped `statusUrl` string from the response body

The `statusUrl` drop was mentioned as "unused by SupplyChainPanel" but
not framed as a contract change. The 202 → 200 shift was not mentioned
at all. This is a same-version (v1 → v1) migration, so external callers
that key off either signal — `response.status === 202` or
`response.body.statusUrl` — silently branch incorrectly.

Evaluated options:
  (a) sebuf per-RPC status-code config — not available. sebuf's
      HttpConfig only models `path` and `method`; no status annotation.
  (b) Bump to scenario/v2 — judged heavier than the break itself for
      a single status-code shift. No in-repo caller uses 202 or
      statusUrl; the docs-level impact is containable.
  (c) Accept the break, document explicitly, partially restore.

Took option (c):

- Restored `statusUrl` in the proto (new field `string status_url = 3`
  on RunScenarioResponse). Server computes
  `/api/scenario/v1/get-scenario-status?jobId=<encoded job_id>` and
  populates it on every successful enqueue. External callers that
  followed this URL keep working unchanged.
- 202 → 200 is not recoverable inside the sebuf generator, so it is
  called out explicitly in two places:
    - docs/api-scenarios.mdx now includes a prominent `<Warning>` block
      documenting the v1→v1 contract shift + the suggested migration
      (branch on response body shape, not HTTP status).
    - RunScenarioResponse proto comment explains why 200 is the new
      success status on enqueue.
  OpenAPI bundle regenerated to reflect the restored statusUrl field.

- Regression test added in tests/scenario-handler.test.mjs pinning
  `statusUrl` to the exact URL-encoded shape — locks the invariant so
  a future proto rename or handler refactor can't silently drop it
  again.

From koala73 review (#3242 second-pass, HIGH new #3).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 1/2): close webhook tenant-isolation gap on shipping/v2 (#3207)

Koala flagged this as a merge blocker in PR #3242 review.

server/worldmonitor/shipping/v2/{register-webhook,list-webhooks}.ts
migrated without reinstating validateApiKey(req, { forceKey: true }),
diverging from both the sibling api/v2/shipping/webhooks/[subscriberId]
routes and the documented "X-WorldMonitor-Key required" contract in
docs/api-shipping-v2.mdx.

Attack surface: the gateway accepts Clerk bearer auth as a pro signal.
A Clerk-authenticated pro user with no X-WorldMonitor-Key reaches the
handler, callerFingerprint() falls back to 'anon', and every such
caller collapses into a shared webhook:owner:anon:v1 bucket. The
defense-in-depth ownerTag !== ownerHash check in list-webhooks.ts
doesn't catch it because both sides equal 'anon' — every Clerk-session
holder could enumerate / overwrite every other Clerk-session pro
tenant's registered webhook URLs.

Fix: reinstate validateApiKey(ctx.request, { forceKey: true }) at the
top of each handler, throwing ApiError(401) when absent. Matches the
sibling routes exactly and the published partner contract.

Tests:
- tests/shipping-v2-handler.test.mjs: two existing "non-PRO → 403"
  tests for register/list were using makeCtx() with no key, which now
  fails at the 401 layer first. Renamed to "no API key → 401
  (tenant-isolation gate)" with a comment explaining the failure mode
  being tested. 18/18 pass.

Verified: typecheck:api, lint:api-contract (no change), lint:boundaries,
lint:rate-limit-policies, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 2/2): restore v1 path aliases on scenario + supply-chain (#3207)

Koala flagged this as a merge blocker in PR #3242 review.

Commits 6 + 7 of #3207 renamed five documented v1 URLs to the sebuf
method-derived paths and deleted the legacy edge-function files:

  POST /api/scenario/v1/run                       → run-scenario
  GET  /api/scenario/v1/status                    → get-scenario-status
  GET  /api/scenario/v1/templates                 → list-scenario-templates
  GET  /api/supply-chain/v1/country-products      → get-country-products
  GET  /api/supply-chain/v1/multi-sector-cost-shock → get-multi-sector-cost-shock

server/router.ts is an exact static-match table (Map keyed on `METHOD
PATH`), so any external caller — docs, partner scripts, grep-the-
internet — hitting the old documented URL would 404 on first request
after merge. Commit 8 (shipping/v2) preserved partner URLs byte-for-
byte; the scenario + supply-chain renames missed that discipline.

Fix: add five thin alias edge functions that rewrite the pathname to
the canonical sebuf path and delegate to the domain [rpc].ts gateway
via a new server/alias-rewrite.ts helper. Premium gating, rate limits,
entitlement checks, and cache-tier lookups all fire on the canonical
path — aliases are pure URL rewrites, not a duplicate handler pipeline.

  api/scenario/v1/{run,status,templates}.ts
  api/supply-chain/v1/{country-products,multi-sector-cost-shock}.ts

Vite dev parity: file-based routing at api/ is a Vercel concern, so the
dev middleware (vite.config.ts) gets a matching V1_ALIASES rewrite map
before the router dispatch.

Manifest: 5 new entries under `deferred` with removal_issue=#3282
(tracking their retirement at the next v1→v2 break). lint:api-contract
stays green (89 files checked, 55 manifest entries validated).

Docs:
- docs/api-scenarios.mdx: migration callout at the top with the full
  old→new URL table and a link to the retirement issue.
- CHANGELOG.md + docs/changelog.mdx: Changed entry documenting the
  rename + alias compat + the 202→200 shift (from commit 23c821a1).

Verified: typecheck:api, lint:api-contract, lint:rate-limit-policies,
lint:boundaries, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-22 09:55:59 +03:00
Elie Habib
4853645d53 fix(brief): switch carousel to @vercel/og on edge runtime (#3210) 
* fix(brief): switch carousel to @vercel/og on edge runtime

Every attempt to ship the Phase 8 Telegram carousel on Vercel's
Node serverless runtime has failed at cold start:

- PR #3174 direct satori + @resvg/resvg-wasm: Vercel edge bundler
  refused the `?url` asset import required by resvg-wasm.
- PR #3174 (fix) direct satori + @resvg/resvg-js native binding:
  Node runtime accepted it, but Vercel's nft tracer does not follow
  @resvg/resvg-js/js-binding.js's conditional
  `require('@resvg/resvg-js-<platform>-<arch>-<libc>')` pattern,
  so the linux-x64-gnu peer package was never bundled. Cold start
  threw MODULE_NOT_FOUND, isolate crashed,
  FUNCTION_INVOCATION_FAILED on every request including OPTIONS,
  and Telegram reported WEBPAGE_CURL_FAILED with no other signal.
- PR #3204 added `vercel.json` `functions.includeFiles` to force
  the binding in, but (a) the initial key was a literal path that
  Vercel micromatch read as a character class (PR #3206 fixed),
  (b) even with the corrected `api/brief/carousel/**` wildcard, the
  function still 500'd across the board. The `functions.includeFiles`
  path appears honored in the deployment manifest but not at runtime
  for this particular native-binding pattern.

Fix: swap the renderer to @vercel/og's ImageResponse, which is
Vercel's first-party wrapper around satori + resvg-wasm with
Vercel-native bundling. Runs on Edge runtime — matches every other
API route in the project. No native binding, no includeFiles, no
nft tracing surprises. Cold start ~300ms, warm ~30ms.

Changes:
- server/_shared/brief-carousel-render.ts: replace renderCarouselPng
  (Uint8Array) with renderCarouselImageResponse (ImageResponse).
  Drop ensureLibs + satori + @resvg/resvg-js dynamic-import dance.
  Keep layout builders (buildCover/buildThreads/buildStory) and
  font loading unchanged — the Satori object trees are
  wire-compatible with ImageResponse.
- api/brief/carousel/[userId]/[issueDate]/[page].ts: flip
  `runtime: 'nodejs'` -> `runtime: 'edge'`. Delegate rendering to
  the renderer's ImageResponse and return it directly; error path
  still 503 no-store so CDN + Telegram don't pin a bad render.
- vercel.json: drop the now-useless `functions.includeFiles` block.
- package.json: drop direct `@resvg/resvg-js` and `satori` deps
  (both now bundled inside @vercel/og).
- tests/deploy-config.test.mjs: replace the native-binding
  regression guards with an assertion that no `functions` block
  exists (with a comment pointing at the skill documenting the
  micromatch gotcha for future routes).
- tests/brief-carousel.test.mjs: updated comment references.

Verified:
- typecheck + typecheck:api clean
- test:data 5814/5814 pass
- node -e test: @vercel/og imports cleanly in Node (tests that
  reach through the renderer file no longer depend on native
  bindings)

Post-deploy validation:
  curl -I -H "User-Agent: TelegramBot (like TwitterBot)" \
    "https://www.worldmonitor.app/api/brief/carousel/<uid>/<slot>/0"
  # Expect: HTTP/2 403 (no token) or 200 (valid token)
  # NOT:    HTTP/2 500 FUNCTION_INVOCATION_FAILED

Then tail Railway digest logs on the next tick; the
`[digest] Telegram carousel 400 ... WEBPAGE_CURL_FAILED` line
should stop appearing, and the 3-image preview should actually land
on Telegram.

* Add renderer smoke test + fix Cache-Control duplication

Reviewer flagged residual risk: no dedicated carousel-route smoke
test for the @vercel/og path. Adds one, and catches a real bug in
the process.

Findings during test-writing:

1. @vercel/og's ImageResponse runs CLEANLY in Node via tsx — the
   comment in brief-carousel.test.mjs saying "we can't test the
   render in Node" was true for direct satori + @resvg/resvg-wasm
   but no longer holds after PR #3210. Pure Node render works
   end-to-end: satori tree-parse, jsdelivr font fetch, resvg-wasm
   init, PNG output. ~850ms first call, ~20ms warm.

2. ImageResponse sets its own default
   `Cache-Control: public, immutable, no-transform, max-age=31536000`.
   Passing Cache-Control via the constructor's headers option
   APPENDS rather than overrides, producing a duplicated
   comma-joined value like
   `public, immutable, no-transform, max-age=31536000, public, max-age=60`
   on the Response. The route handler was doing exactly this via
   extraHeaders. Fix: drop our Cache-Control override and rely on
   @vercel/og's 1-year immutable default — envelope is only
   immutable for its 7d Redis TTL so the effective ceiling is 7d
   anyway (after that the route 404s before render).

Changes:

- tests/brief-carousel.test.mjs: 6 new assertions under
  `renderCarouselImageResponse`:
    * renders cover / threads / story pages, each returning a
      valid PNG (magic bytes + size range)
    * rejects a structurally empty envelope
    * threads non-cache extraHeaders onto the Response
    * pins @vercel/og's Cache-Control default so it survives
      caller-supplied Cache-Control overrides (regression guard
      for the bug fixed in this commit)
- api/brief/carousel/[userId]/[issueDate]/[page].ts: remove the
  stacked Cache-Control; lean on @vercel/og default. Drop the now-
  unused `PAGE_CACHE_TTL` constant. Comment explains why.

Verified:
- test:data 5820/5820 pass (was 5814, +6 smoke)
- typecheck + typecheck:api clean
- Render smoke: cover 825ms / threads 23ms / story 16ms first run
  (wasm init dominates first render)
 2026-04-19 15:18:12 +04:00
Elie Habib
122204f691 feat(brief): Phase 8 — Telegram carousel images via Satori + resvg-wasm (#3174) 
* feat(brief): Phase 8 — Telegram carousel images via Satori + resvg-wasm

Implements the Phase 8 carousel renderer (Option B): server-side PNG
generation in a Vercel edge function using Satori (JSX to SVG) +
@resvg/resvg-wasm (SVG to PNG). Zero new Railway infra, zero
Chromium, same edge runtime that already serves the magazine HTML.

Files:

  server/_shared/brief-carousel-render.ts (new)
    Pure renderer: (BriefEnvelope, CarouselPage) -> Uint8Array PNG.
    Three layouts (cover/threads/story), 1200x630 OG size.
    Satori + resvg + WASM are lazy-imported so Node tests don't trip
    over '?url' asset imports and the 800KB wasm doesn't ship in
    every bundle. Font: Noto Serif regular, fetched once from Google
    Fonts and memoised on the edge isolate.

  api/brief/carousel/[userId]/[issueDate]/[page].ts (new)
    Public edge function reusing the magazine route's HMAC token —
    same signer, same (userId, issueDate) binding, so one token
    unlocks magazine HTML AND all three carousel images. Returns
    image/png with 7d immutable cache headers. 404 on invalid page
    index, 403 on bad token, 404 on Redis miss, 503 on missing
    signing secret. Render failure falls back to a 1x1 transparent
    PNG so Telegram's sendMediaGroup doesn't 500 the brief.

  scripts/seed-digest-notifications.mjs
    carouselUrlsFrom(magazineUrl) derives the 3 signed carousel
    URLs from the already-signed magazine URL. sendTelegramBriefCarousel
    calls Telegram's sendMediaGroup with those URLs + short caption.
    Runs before the existing sendTelegram(text) so the carousel is
    the header and the text the body — long-form stories remain
    forwardable as text. Best-effort: carousel failure doesn't
    block text delivery.

  package.json + package-lock.json
    satori ^0.10.14 + @resvg/resvg-wasm ^2.6.2.

Tests (tests/brief-carousel.test.mjs, 9 cases):
  - pageFromIndex mapping + out-of-range
  - carouselUrlsFrom: valid URL, localhost origin preserved, missing
    token, wrong path, invalid issueDate, garbage input
  - Drift guard: cron must still declare the same helper + template
    string. If it drifts, test fails with a pointer to move the impl
    into a shared module.

PNG render itself isn't unit-tested — Satori + WASM need a
browser/edge runtime. Covered by smoke validation step in the
deploy monitoring plan.

Both tsconfigs typecheck clean. 152/152 brief tests pass.

Scope boundaries (deferred):
  - Slack + Discord image attachments (different payload shapes)
  - notification-relay.cjs brief_ready dispatch (real-time route)
  - Redis caching of rendered PNG (edge Cache-Control is enough for
    MVP)

* fix(brief): address two P1 review findings on Phase 8 carousel

P1-A: 200 placeholder PNG cached 7d on render failure.
Route config said runtime: 'edge' but a comment contradicted it
claiming Node semantics. More importantly, any render/init failure
(WASM load, Satori, Google Fonts) was converted to a 1x1 transparent
PNG returned with Cache-Control: public, max-age=7d, immutable.
Telegram's media fetcher and Vercel's CDN would cache that blank
for the full brief TTL per chat message — one cold-start mismatch
= every reader of that brief sees blank carousel previews for a
week.

Fix: deleted errorPng(). Render failure now returns 503 with
Cache-Control: no-store. sendMediaGroup fails cleanly for that
carousel (the digest cron already treats it as best-effort and
still sends the long-form text message), next cron tick re-renders
from a fresh isolate. Self-healing across ticks. Contradictory
comment about Node runtime removed.

P1-B: Google Fonts as silent hard dependency.
The renderer claimed 'safe embedded/fallback path' in comments but
no fallback existed. loadFont() fetches Noto Serif from gstatic.com
and rethrows on any failure. Combined with P1-A's old 200-cache-7d
path, a transient CDN blip would lock in a blank carousel for a
week.

Fix: updated comments to honestly declare the CDN dependency plus
document the self-healing semantics now that P1-A's fix no longer
caches the failure. If Google Fonts reliability becomes a problem,
swap the fetch for a bundled base64 TTF — noted as the escape hatch.

Tests (tests/brief-carousel.test.mjs): 2 new regression cases.
11/11 carousel tests pass. Both tsconfigs typecheck clean locally.

Note on currently-red CI: failures are NOT typecheck errors — npm
ci dies fetching libvips for sharp (504 Gateway Time-out from
GitHub releases). sharp is a transitive dep via @xenova/transformers,
pre-existing, not touched by this PR. Transient infra flake.

* fix(brief): switch carousel to Node + @resvg/resvg-js (fixes deploy block)

Vercel edge bundler fails the carousel deploy with:
  'Edge Function is referencing unsupported modules:
   @resvg/resvg-wasm/index_bg.wasm?url'

The ?url asset-import syntax is a Vite-ism that Vercel's edge
bundler doesn't resolve. Two ways out: find a Vercel-blessed edge
WASM import incantation, or switch to Node runtime with the native
@resvg/resvg-js binding. The second is simpler, faster per request,
and avoids the whole WASM-in-edge-bundler rabbit hole.

Changes:
  - package.json: @resvg/resvg-wasm -> @resvg/resvg-js ^2.6.2
  - api/brief/carousel/.../[page].ts: runtime 'edge' -> 'nodejs20.x'
  - server/_shared/brief-carousel-render.ts: drop initWasm path,
    dynamic-import resvg-js in ensureLibs(). Satori and resvg load
    in parallel via Promise.all, shaving ~30ms off cold start.

Also addresses the P2 finding from review: the old ensureLibsAndWasm
had a concurrent-cold-start race where two callers could reach
'await initWasm()' simultaneously. Replaced the boolean flag with a
shared _libsLoadPromise so concurrent callers await the same load.
On failure the promise resets so the NEXT request retries rather
than poisoning the isolate for its lifetime.

Cold start ~700ms (Satori + resvg-js native init), warm ~40ms.
Carousel images are not latency-critical — fetched by Telegram's
media service, CDN-cached 7d.

Both tsconfigs typecheck clean. 11/11 carousel tests pass.

* fix(brief): carousel runtime = 'nodejs' (was 'nodejs20.x', rejected by Vercel)

Vercel's functions config validator rejects 'nodejs20.x' at deploy
time:

  unsupported "runtime" value in config: "nodejs20.x"
  (must be one of: ["edge","experimental-edge","nodejs"])

The Node version comes from the project's default (currently Node 20
via package.json engines + Vercel project settings), not from the
runtime string. Use 'nodejs' — unversioned — and let the platform
resolve it.

11/11 carousel tests pass.

* fix(brief): swap carousel font from woff2 to woff (Satori can't parse woff2)

Review on PR #3174: the FONT_URL was pointing at a gstatic.com woff2
file. Satori parses ttf / otf / woff v1 — NOT woff2. Every render
was about to throw on font decode, the route would return 503, and
the carousel would never deliver a single image.

Fix: point FONT_URL at @fontsource's Noto Serif Latin 400 WOFF v1
via jsdelivr. WOFF v1 is a TrueType wrapper that Satori parses
natively (verified: file says 'Web Open Font Format, TrueType,
version 1.1'). Same cold-start semantics as before — one fetch per
warm isolate, memoised.

Regression test: asserts FONT_URL ends in ttf/otf/woff and explicitly
rejects any .woff2 suffix. A future swap that silently reintroduces
woff2 now fails CI loudly instead of shipping a permanently-broken
renderer.

12/12 carousel tests pass. Both tsconfigs typecheck clean.
 2026-04-18 20:27:41 +04:00
Elie Habib
30ddad28d7 fix(seeds): upstream API drift — SPDR XLSX + IMF IRFCL + IMF-External BX/BM drop (#3076) 
* fix(seeds): gold-etf XLSX migration, IRFCL dataflow, imf-external BX/BM drop

Three upstream-drift regressions caught from the market-backup + imf-extended
bundle logs. Root causes validated by live API probes before coding.

1. seed-gold-etf-flows: SPDR /assets/dynamic/GLD/GLD_US_archive_EN.csv now
   silently returns a PDF (Content-Type: application/pdf) — site migrated
   to api.spdrgoldshares.com/api/v1/historical-archive which serves XLSX.
   Swapped the CSV parser for an exceljs-based XLSX parser. Adds
   browser-ish Origin/Referer headers (SPDR swaps payload for PDF
   without them) and a Content-Type guard. Column layout: Date | Closing |
   ... | Tonnes | Total NAV USD.

2. seed-gold-cb-reserves: PR #3038 shipped with IMF.STA/IFS dataflow and
   3-segment key M..<indicator> — both wrong. IFS isn't exposed on
   api.imf.org (HTTP 204). Gold-reserves data lives under IMF.STA/IRFCL
   with 4 dimensions (COUNTRY.INDICATOR.SECTOR.FREQUENCY). Verified live:
   *.IRFCLDT1_IRFCL56_FTO.*.M returns 111 series. Switched to IRFCL +
   IRFCLDT1_IRFCL56_FTO (fine troy ounces) and fallbacks. The
   valueIsOunces flag now matches _FTO suffix (keeps legacy _OZT/OUNCE
   detection for backward compat).

3. seed-imf-external: BX/BM (export/import LEVELS, USD) WEO coverage
   collapsed to ~10 countries in late 2026 — the seeder's >=190-country
   validate floor was failing every run. Dropped BX/BM from fetch + join;
   kept BCA (~209) / TM_RPCH (~189) / TX_RPCH (~190). exportsUsd /
   importsUsd / tradeBalanceUsd fields kept as explicit null so consumers
   see a deliberate gap. validate floor lowered to 180 (BCA∪TM∪TX union).

Tests: 32/32 pass. Rewrote gold-etf tests to use synthetic XLSX fixtures
(exceljs resolved from scripts/package.json since repo root doesn't have
it). Updated imf-external tests for the new indicator set + null BX/BM
contract + 180-country validate threshold.

* fix(mcp): update get_country_macro description after BX/BM drop

Consumer-side catch during PR #3076 validation: the MCP tool description
still promised 'exports, imports, trade balance' fields that the seeder
fix nulls out. LLM consumers would be directed to exportsUsd/importsUsd/
tradeBalanceUsd fields that always return null since seed-imf-external
dropped BX/BM (WEO coverage collapsed to ~10 countries).

Updated description to list only the indicators actually populated
(currentAccountUsd, importVolumePctChg, exportVolumePctChg) with an
explicit note about the null trade-level fields so LLMs don't attempt
to use them.

* fix(gold-cb-reserves): compute real pctOfReserves + add exceljs to root

Follow-up to #3076 review.

1. pctOfReserves was hardcoded to 0 with a "IFS doesn't give us total
   reserves" comment. That was a lazy limitation claim — IMF IRFCL DOES
   expose total official reserve assets as IRFCLDT1_IRFCL65_USD parallel
   to the gold USD series IRFCLDT1_IRFCL56_USD. fetchCbReserves now
   pulls all three indicators (primary FTO tonnage + the two USD series)
   via Promise.allSettled and passes the USD pair to buildReservesPayload
   so it can compute the true gold share per country. Falls back to 0
   only when the denominator is genuinely missing for that country
   (IRFCL coverage: 114 gold_usd, 96 total_usd series; ~15% of holders
   have no matched-month denominator). 3-month lookback window absorbs
   per-country reporting lag.

2. CI fix: tests couldn't find exceljs because scripts/package.json is
   not workspace-linked to the repo root — CI runs `npm ci` at root
   only. Added exceljs@^4.4.0 as a root devDependency. Runtime seeder
   continues to resolve it from scripts/node_modules via Node's upward
   module resolution.

3 new tests cover pct computation, missing-denominator fallback, and
the 3-month lookback window.
 2026-04-14 08:19:47 +04:00
Elie Habib
79d5cf8bb8 security: upgrade @anthropic-ai/sdk 0.79.0 -> 0.82.0 + audit fix (#2846) 
Closes GHSA-54m3-95j9-v89j (HIGH) for @anthropic-ai/sdk.
npm audit fix also resolves lodash/lodash-es HIGH CVEs
(GHSA-r5fr-rjxr-66jc, GHSA-f23m-r3pf-42rh).

Co-authored-by: Winston9520 <143687718+Winston9520@users.noreply.github.com>
 2026-04-09 08:50:33 +04:00
Chris Chen
2ad249dbab fix(deckgl): prevent crash on Intel integrated GPUs when satellite imagery layer fails (#2696) 
* fix(deckgl): prevent crash on Intel integrated GPUs when satellite imagery layer fails

- Update deck.gl 9.2.6 → 9.2.11 (latest patch with improved Intel GPU workarounds)
- Extend onError handler to catch satellite-imagery-layer shader compilation failures
- Skip satellite imagery layer gracefully when WebGL shader compilation fails (prevents app crash)
- Follows same graceful-degradation pattern used for apt-groups-layer errors
- Add showLayerWarning when satellite layer fails so users understand why imagery disappeared

Fixes #2518

* fix(deckgl): address greptile-apps P0/P1/P2 review comments on PR #2696

P0: Replace out-of-scope WARN_THRESHOLD with debouncedRebuildLayers()
     WARN_THRESHOLD was a local const inside enforceLayerLimit(), not
     accessible in the onError callback — would cause ReferenceError at
     runtime on Intel GPU failures.

P1: Replace misleading showLayerWarning() with GPU-specific console.warn
     showLayerWarning() tells users they have too many layers active,
     which is deeply misleading for a GPU shader compilation failure.
     Now logs a descriptive GPU incompatibility message instead.

P2: Trigger immediate layer stack rebuild after satellite layer failure
     Added debouncedRebuildLayers() call so the failed layer is removed
     from deck.gl's render list immediately rather than waiting for the
     next moveend/zoomend event.

Fixes greptile-apps review comments:
  - 3036106846 (P0): WARN_THRESHOLD not in scope
  - 3036106896 (P1): showLayerWarning() wrong message
  - 3036106953 (P2): layer not immediately removed

* fix(deckgl): rebuild layers synchronously on satellite shader failure

The debounced rebuild leaves a 150ms window where the broken layer
is still in deck.gl's picking pipeline. A hover/click during that
window can trigger the same crash this PR prevents. Use immediate
setProps instead.

---------

Co-authored-by: fuleinist <fuleinist@gmail.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
 2026-04-06 15:48:21 +04:00
Elie Habib
50626a40c7 chore: bump version to 2.6.7 (#2637) 
* chore: bump version to 2.6.7

* chore: sync Cargo.lock to 2.6.7 (was stuck at 2.6.5)

* ci: skip lint/test/typecheck for non-code PRs (docs, Tauri, version bumps)

Added paths-ignore to lint-code, test, and typecheck workflows so they
don't run when only markdown, docs, src-tauri config, or desktop build
files change. Push to main still runs unconditionally.
 2026-04-03 07:37:45 +04:00
Sebastien Melki
9893bb1cf2 feat: Dodo Payments integration + entitlement engine & webhook pipeline (#2024) 
* feat(14-01): install @dodopayments/convex and register component

- Install @dodopayments/convex@0.2.8 with peer deps satisfied
- Create convex/convex.config.ts with defineApp() and dodopayments component
- Add TODO for betterAuth registration when PR #1812 merges

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(14-01): extend schema with 6 payment tables for Dodo integration

- Add subscriptions table with status enum, indexes, and raw payload
- Add entitlements table (one record per user) with features blob
- Add customers table keyed by userId with optional dodoCustomerId
- Add webhookEvents table for full audit trail (retained forever)
- Add paymentEvents table for billing history (charge/refund)
- Add productPlans table for product-to-plan mapping in DB
- All existing tables (registrations, contactMessages, counters) unchanged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(14-01): add auth stub, env helper, and Dodo env var docs

- Create convex/lib/auth.ts with resolveUserId (returns test-user-001 in dev)
  and requireUserId (throws on unauthenticated) as sole auth entry points
- Create convex/lib/env.ts with requireEnv for runtime env var validation
- Append DODO_API_KEY, DODO_WEBHOOK_SECRET, DODO_PAYMENTS_WEBHOOK_SECRET,
  and DODO_BUSINESS_ID to .env.example with setup instructions
- Document dual webhook secret naming (library vs app convention)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(14-02): add seed mutation for product-to-plan mappings

- Idempotent upsert mutation for 5 Dodo product-to-plan mappings
- Placeholder product IDs to be replaced after Dodo dashboard setup
- listProductPlans query for verification and downstream use

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(14-02): populate seed mutation with real Dodo product IDs

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(15-02): add plan-to-features entitlements config map

- Define PlanFeatures type with 5 feature dimensions
- Add PLAN_FEATURES config for 6 tiers (free through enterprise)
- Export getFeaturesForPlan helper with free-tier fallback
- Export FREE_FEATURES constant for default entitlements

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(15-01): add webhook HTTP endpoint with signature verification

- Custom httpAction verifying Dodo webhook signatures via @dodopayments/core
- Returns 400 for missing headers, 401 for invalid signature, 500 for processing errors
- HTTP router at /dodopayments-webhook dispatches POST to webhook handler
- Synchronous processing before 200 response (within Dodo 15s timeout)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(15-02): add subscription lifecycle handlers and entitlement upsert

- Add upsertEntitlements helper (creates/updates per userId, no duplicates)
- Add isNewerEvent guard for out-of-order webhook rejection
- Add handleSubscriptionActive (creates subscription + entitlements)
- Add handleSubscriptionRenewed (extends period + entitlements)
- Add handleSubscriptionOnHold (pauses without revoking entitlements)
- Add handleSubscriptionCancelled (preserves entitlements until period end)
- Add handleSubscriptionPlanChanged (updates plan + recomputes entitlements)
- Add handlePaymentEvent (records charge events for succeeded/failed)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(15-01): add idempotent webhook event processor with dispatch skeleton

- processWebhookEvent internalMutation with idempotency via by_webhookId index
- Switch dispatch for 7 event types: 5 subscription + 2 payment events
- Stub handlers log TODO for each event type (to be implemented in Plan 03)
- Error handling marks failed events and re-throws for HTTP 500 + Dodo retry

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs(15-01): complete webhook endpoint plan

- Update auto-generated api.d.ts with new payment module types
- SUMMARY, STATE, and ROADMAP updated (.planning/ gitignored)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(15-03): wire subscription handlers into webhook dispatch

- Replace 6 stub handler functions with imports from subscriptionHelpers
- All 7 event types (5 subscription + 2 payment) dispatch to real handlers
- Error handling preserves failed event status in webhookEvents table
- Complete end-to-end pipeline: HTTP action -> mutation -> handler functions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(15-04): install convex-test, vitest, and edge-runtime; configure vitest

- Add convex-test, vitest, @edge-runtime/vm as dev dependencies
- Create vitest.config.mts scoped to convex/__tests__/ with edge-runtime environment
- Add test:convex and test:convex:watch npm scripts

* test(15-04): add 10 contract tests for webhook event processing pipeline

- Test all 5 subscription lifecycle events (active, renewed, on_hold, cancelled, plan_changed)
- Test both payment events (succeeded, failed)
- Test deduplication by webhook-id (same id processed only once)
- Test out-of-order event rejection (older timestamp skipped)
- Test subscription reactivation (cancelled -> active on same subscription_id)
- Verify entitlements created/updated with correct plan features

* fix(15-04): exclude __tests__ from convex typecheck

convex-test uses Vite-specific import.meta.glob and has generic type
mismatches with tsc. Tests run correctly via vitest; excluding from
convex typecheck avoids false positives.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(16-01): add tier levels to PLAN_FEATURES and create entitlement query

- Add tier: number to PlanFeatures type (0=free, 1=pro, 2=api, 3=enterprise)
- Add tier values to all plan entries in PLAN_FEATURES config
- Create convex/entitlements.ts with getEntitlementsForUser public query
- Free-tier fallback for missing or expired entitlements

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(16-01): create Redis cache sync action and wire upsertEntitlements

- Create convex/payments/cacheActions.ts with syncEntitlementCache internal action
- Wire upsertEntitlements to schedule cache sync via ctx.scheduler.runAfter(0, ...)
- Add deleteRedisKey() to server/_shared/redis.ts for explicit cache invalidation
- Redis keys use raw format (entitlements:{userId}) with 1-hour TTL
- Cache write failures logged but do not break webhook pipeline

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(16-02): add entitlement enforcement to API gateway

- Create entitlement-check middleware with Redis cache + Convex fallback
- Replace PREMIUM_RPC_PATHS boolean Set with ENDPOINT_ENTITLEMENTS tier map
- Wire checkEntitlement into gateway between API key and rate limiting
- Add raw parameter to setCachedJson for user-scoped entitlement keys
- Fail-open on missing auth/cache failures for graceful degradation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(16-03): create frontend entitlement service with reactive ConvexClient subscription

- Add VITE_CONVEX_URL to .env.example for frontend Convex access
- Create src/services/entitlements.ts with lazy-loaded ConvexClient
- Export initEntitlementSubscription, onEntitlementChange, getEntitlementState, hasFeature, hasTier, isEntitled
- ConvexClient only loaded when userId available and VITE_CONVEX_URL configured
- Graceful degradation: log warning and skip when Convex unavailable

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(16-04): add 6 contract tests for Convex entitlement query

- Free-tier defaults for unknown userId
- Active entitlements for subscribed user
- Free-tier fallback for expired entitlements
- Correct tier mapping for api_starter and enterprise plans
- getFeaturesForPlan fallback for unknown plan keys

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(16-04): add 6 unit tests for gateway entitlement enforcement

- getRequiredTier: gated vs ungated endpoint tier lookup
- checkEntitlement: ungated pass-through, missing userId graceful degradation
- checkEntitlement: 403 for insufficient tier, null for sufficient tier
- Dependency injection pattern (_testCheckEntitlement) for clean testability
- vitest.config.mts include expanded to server/__tests__/

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(17-01): create Convex checkout session action

- DodoPayments component wraps checkout with server-side API key
- Accepts productId, returnUrl, discountCode, referralCode args
- Always enables discount code input (PROMO-01)
- Forwards affiliate referral as checkout metadata (PROMO-02)
- Dark theme customization for checkout overlay

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(17-03): create PricingSection component with tier cards and billing toggle

- 4 tiers: Free, Pro, API, Enterprise with feature comparison
- Monthly/annual toggle with "Save 17%" badge for Pro
- Checkout buttons using Dodo static payment links
- Pro tier visually highlighted with green border and "Most Popular" badge
- Staggered entrance animations via motion
- Referral code forwarding via refCode prop

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* refactor(17-01): extract shared ConvexClient singleton, refactor entitlements

- Create src/services/convex-client.ts with getConvexClient() and getConvexApi()
- Lazy-load ConvexClient via dynamic import to preserve bundle size
- Refactor entitlements.ts to use shared client instead of inline creation
- Both checkout and entitlement services will share one WebSocket connection

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(17-03): integrate PricingSection into App.tsx with referral code forwarding

- Import and render PricingSection between EnterpriseShowcase and PricingTable
- Pass refCode from getRefCode() URL param to PricingSection for checkout link forwarding
- Update navbar CTA and TwoPathSplit Pro CTA to anchor to #pricing section
- Keep existing waitlist form in Footer for users not ready to buy
- Build succeeds with no new errors

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update generated files after main merge and pro-test build

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: prefix unused ctx param in auth stub to pass typecheck

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(17-04): add 4 E2E contract tests for checkout-to-entitlement flow

- Test product plan seeding and querying (5 plans verified)
- Test pro_monthly checkout -> webhook -> entitlements (tier=1, no API)
- Test api_starter checkout -> webhook -> entitlements (tier=2, apiAccess)
- Test expired entitlements fall back to free tier (tier=0)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(17-02): install dodopayments-checkout SDK and create checkout overlay service

- Install dodopayments-checkout@1.8.0 overlay SDK
- Create src/services/checkout.ts with initCheckoutOverlay, openCheckout, startCheckout, showCheckoutSuccess
- Dark theme config matching dashboard aesthetic (green accent, dark bg)
- Lazy SDK initialization on first use
- Fallback to /pro page when Convex is unavailable

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(17-02): wire locked panel CTAs and post-checkout return handling

- Create src/services/checkout-return.ts for URL param detection and cleanup
- Update Panel.ts showLocked() CTA to trigger Dodo overlay checkout (web path)
- Keep Tauri desktop path opening URL externally
- Add handleCheckoutReturn() call in PanelLayoutManager constructor
- Initialize checkout overlay with success banner callback
- Dynamic import of checkout module to avoid loading until user clicks

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(17-02): add Upgrade to Pro section in UnifiedSettings modal

- Add upgrade section at bottom of settings tab with value proposition
- Wire CTA button to open Dodo checkout overlay via dynamic import
- Close settings modal before opening checkout overlay
- Tauri desktop fallback to external URL
- Conditionally show "You're on Pro" when user has active entitlement

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: remove unused imports in entitlement-check test to pass typecheck:api

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(17-01): guard missing DODO_PAYMENTS_API_KEY with warning instead of silent undefined

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(17-01): use DODO_API_KEY env var name matching Convex dashboard config

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(17-02): add Dodo checkout domains to CSP frame-src directive

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(17-03): use test checkout domain for test-mode Dodo products

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(18-01): shared DodoPayments config and customer upsert in webhook

- Create convex/lib/dodo.ts centralizing DodoPayments instance and API exports
- Refactor checkout.ts to import from shared config (remove inline instantiation)
- Add customer record upsert in handleSubscriptionActive for portal session support

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(18-01): billing queries and actions for subscription management

- Add getSubscriptionForUser query (plan status, display name, renewal date)
- Add getCustomerByUserId and getActiveSubscription internal queries
- Add getCustomerPortalUrl action (creates Dodo portal session via SDK)
- Add changePlan action (upgrade/downgrade with proration via Dodo SDK)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(18-02): add frontend billing service with reactive subscription watch

- SubscriptionInfo interface for plan status display
- initSubscriptionWatch() with ConvexClient onUpdate subscription
- onSubscriptionChange() listener pattern with immediate fire for late subscribers
- openBillingPortal() with Dodo Customer Portal fallback
- changePlan() with prorated_immediately proration mode

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(18-02): add subscription status display and Manage Billing button to settings

- Settings modal shows plan name, status badge, and renewal date for entitled users
- Status-aware colors: green (active), yellow (on_hold), red (cancelled/expired)
- Manage Billing button opens Dodo Customer Portal via billing service
- initSubscriptionWatch called at dashboard boot alongside entitlements
- Import initPaymentFailureBanner for Task 3 wiring

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(18-02): add persistent payment failure banner for on_hold subscriptions

- Red fixed-position banner at top of dashboard when subscription is on_hold
- Update Payment button opens Dodo billing portal
- Dismiss button with sessionStorage persistence (avoids nagging in same session)
- Auto-removes when subscription returns to active (reactive via Convex)
- Event listeners attached directly to DOM (not via debounced setContent)
- Wired into panel-layout constructor alongside subscription watch

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address code review — identity bridge, entitlement gating, fail-closed, env hygiene

P0: Checkout-to-user identity bridge
- Pass userId as metadata.wm_user_id in checkout sessions
- Webhook resolveUserId: try metadata first, then customer table, then dev-only fallback
- Fail closed in production when no user identity can be resolved

P0: Unify premium gating to read Dodo entitlements
- data-loader.ts: hasPremiumAccess() checks isEntitled() || API key
- panels.ts: isPanelEntitled checks isEntitled() before API key fallback
- panel-layout.ts: reload on entitlement change to unlock panels

P1: Fail closed on unknown product IDs
- resolvePlanKey throws on unmapped product (webhook retries)
- getFeaturesForPlan throws on unknown planKey

P1: Env var hygiene
- Canonical DODO_API_KEY (no dual-name fallback in dodo.ts)
- console.error on missing key instead of silent empty string

P1: Fix test suite scheduled function errors
- Guard scheduler.runAfter with UPSTASH_REDIS_REST_URL check
- Tests skip Redis cache sync, eliminating convex-test write errors

P2: Webhook rollback durability
- webhookMutations: return error instead of rethrow (preserves audit row)
- webhookHandlers: check mutation return for error indicator

P2: Product ID consolidation
- New src/config/products.ts as single source of truth
- Panel.ts and UnifiedSettings.ts import from shared config

P2: ConvexHttpClient singleton in entitlement-check.ts
P2: Concrete features validator in schema (replaces v.any())
P2: Tests seed real customer mapping (not fallback user)

P3: Narrow eslint-disable to typed interfaces in subscriptionHelpers
P3: Real ConvexClient type in convex-client.ts
P3: Better dev detection in auth.ts (CONVEX_IS_DEV)
P3: Add VITE_DODO_ENVIRONMENT to .env.example

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(payments): security audit hardening — auth gates, webhook retry, transaction safety

- Gate all public billing/checkout endpoints with resolveUserId(ctx) auth check
- Fix webhook retry: record events after processing, not before; delete failed events on retry
- Fix transaction atomicity: let errors propagate so Convex rolls back partial writes
- Fix isDevDeployment to use CONVEX_IS_DEV (same as lib/auth.ts)
- Add missing handlers: subscription.expired, refund.*, dispute.*
- Fix toEpochMs silent fallback — now warns on missing billing dates
- Use validated payload directly instead of double-parsing webhook body
- Fix multi-sub query to prioritize active > on_hold > cancelled > expired
- Change .unique() to .first() on customer lookups (defensive against duplicates)
- Update handleSubscriptionActive to patch planKey/dodoProductId on existing subs
- Frontend: portal URL validation, getProWidgetKey(), subscription cleanup on destroy
- Make seedProductPlans internalMutation, console.log → console.warn for ops signals

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address code review — identity bridge, entitlement gating, fail-safe dev detection

P0: convex/lib/auth.ts + subscriptionHelpers.ts — remove CONVEX_CLOUD_URL
heuristic that could treat production as dev. Now uses ctx.auth.getUserIdentity()
as primary auth with CONVEX_IS_DEV-only dev fallback.

P0: server/gateway.ts + auth-session.ts — add bearer token (Clerk JWT) support
for tier-gated endpoints. Authenticated users bypass API key requirement; userId
flows into x-user-id header for entitlement check. Activated by setting
CLERK_JWT_ISSUER_DOMAIN env var.

P1: src/services/user-identity.ts — centralized getUserId() replacing scattered
getProWidgetKey() calls in checkout.ts, billing.ts, panel-layout.ts.

P2: src/App.ts — premium panel prime/refresh now checks isEntitled() alongside
WORLDMONITOR_API_KEY so Dodo-entitled web users get data loading.

P2: convex/lib/dodo.ts + billing.ts — move Dodo SDK config from module scope
into lazy/action-scoped init. Missing DODO_API_KEY now throws at action boundary
instead of silently capturing empty string.

Tests: webhook test payloads now include wm_user_id metadata (production path).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address P0 access control + P1 identity bridge + P1 entitlement reload loop

P0: Remove userId from public billing function args (getSubscriptionForUser,
getCustomerPortalUrl, changePlan) — use requireUserId(ctx) with no fallback
to prevent unauthenticated callers from accessing arbitrary user data.

P1: Add stable anonymous ID (wm-anon-id) in user-identity.ts so
createCheckout always passes wm_user_id in metadata. Breaks the infinite
webhook retry loop for brand-new purchasers with no auth/localStorage identity.

P1: Skip initial entitlement snapshot in onEntitlementChange to prevent
reload loop for existing premium users whose shouldUnlockPremium() is
already true from legacy signals (API key / wm-pro-key).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address P1 billing flow + P1 anon ID claim path + P2 daily-market-brief

P1 — Billing functions wired end-to-end for browser sessions:
- getSubscriptionForUser, getCustomerPortalUrl, changePlan now accept
  userId from args (matching entitlements.ts pattern) with auth-first
  fallback. Once Clerk JWT is wired into ConvexClient.setAuth(), the
  auth path will take precedence automatically.
- Frontend billing.ts passes userId from getUserId() on all calls.
- Subscription watch, portal URL, and plan change all work for browser
  users with anon IDs.

P1 — Anonymous ID → account claim path:
- Added claimSubscription(anonId) mutation to billing.ts — reassigns
  subscriptions, entitlements, customers, and payment events from an
  anonymous browser ID to the authenticated user.
- Documented the anon ID limitation in user-identity.ts with the
  migration plan (call claimSubscription on first Clerk session).
- Created follow-up issue #2078 for the full claim/migration flow.

P2 — daily-market-brief added to hasPremiumAccess() block in
data-loader.ts loadAllData() so it loads on general data refresh
paths (was only in primeVisiblePanelData startup path).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: P0 lock down billing write actions + P2 fix claimSubscription logic

P0 — Billing access control locked down:
- getCustomerPortalUrl and changePlan converted to internalAction — not
  callable from the browser, closing the IDOR hole on write paths.
- getSubscriptionForUser stays as a public query with userId arg (read-only,
  matching the entitlements.ts pattern — low risk).
- Frontend billing.ts: portal opens generic Dodo URL, changePlan returns
  "not available" stub. Both will be promoted once Clerk auth is wired
  into ConvexClient.setAuth().

P2 — claimSubscription merge logic fixed:
- Entitlement comparison now uses features.tier first, breaks ties with
  validUntil (was comparing only validUntil which could downgrade tiers).
- Added Redis cache invalidation after claim: schedules
  deleteEntitlementCache for the stale anon ID and syncEntitlementCache
  for the real user ID.
- Added deleteEntitlementCache internal action to cacheActions.ts.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(billing): strip Dodo vendor IDs from public query response

Remove dodoSubscriptionId and dodoProductId from getSubscriptionForUser
return — these vendor-level identifiers aren't used client-side and
shouldn't be exposed over an unauthenticated fallback path.

Addresses koala73's Round 5 P1 review on #2024.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(billing): address koala review cleanup items (P2/P3)

- Remove stale dodoSubscriptionId/dodoProductId from SubscriptionInfo
  interface (server no longer returns them)
- Remove dead `?? crypto.randomUUID()` fallback in checkout.ts
  (getUserId() always returns a string via getOrCreateAnonId())
- Remove unused "failed" status variant and errorMessage from
  webhookEvents schema (no code path writes them)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add missing isProUser import and allow trusted origins for tier-gated endpoints

The typecheck failed because isProUser was used in App.ts but never imported.
The unit test failed because the gateway forced API key validation for
tier-gated endpoints even from trusted browser origins (worldmonitor.app),
where the client-side isProUser() gate controls access instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(gateway): require credentials for premium endpoints regardless of origin

Origin header is spoofable — it cannot be a security boundary. Premium
endpoints now always require either an API key or a valid bearer token
(via Clerk session). Authenticated users (sessionUserId present) bypass
the API key check; unauthenticated requests to tier-gated endpoints get 401.

Updated test to assert browserNoKey → 401 instead of 200.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tests): add catch-all route to legacy endpoint allowlist

The [[...path]].js Vercel catch-all route (domain gateway entry point)
was missing from ALLOWED_LEGACY_ENDPOINTS in the edge function tests.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* revert: remove [[...path]].js from legacy endpoint allowlist

This file is Vercel-generated and gitignored — it only exists locally,
not in the repo. Adding it to the allowlist caused CI to fail with
"stale entry" since the file doesn't exist in CI.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(payment): apply design system to payment UI (light/dark mode)

- checkout.ts: add light themeConfig for Dodo overlay + pass theme flag
  based on current document.dataset.theme; previously only dark was
  configured so light-mode users got Dodo's default white UI
- UnifiedSettings: replace hardcoded dark hex values (#1a1a1a, #323232,
  #fff, #909090) in upgrade section with CSS var-driven classes so the
  panel respects both light and dark themes
- main.css: add .upgrade-pro-section / .upgrade-pro-cta / .manage-billing-btn
  classes using var(--green), var(--bg), var(--surface), var(--border), etc.

* fix(checkout): remove invalid theme prop from CheckoutOptions

* fix: regenerate package-lock.json with npm 10 (matches CI Node 22)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(gateway): enforce pro role check for authenticated free users on premium paths

Free bearer token holders with a valid session bypassed PREMIUM_RPC_PATHS
because sessionUserId being set caused forceKey=false, skipping the role
check entirely. Now explicitly checks bearer role after API key gate.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove unused getSecretState import from data-loader

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: gate Dodo Payments init behind isProUser() (same as Clerk)

Entitlement subscription, subscription watch, checkout overlay, and
payment banners now only initialize for isProUser() — matching the
Clerk auth gate so only wm-pro-key / wm-widget-key holders see it.

Also consolidates inline isEntitled()||getSecretState()||role checks
in App.ts to use the centralized hasPremiumAccess() from panel-gating.

Both gates (Clerk + Dodo) to be removed when ready for all users.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* ci: retrigger workflows

* chore: retrigger CI

* fix(redis): use POST method in deleteRedisKey for consistency

All other write helpers use POST; DEL was implicitly using GET via fetch default.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(quick-5): resolve all P1 security issues from koala73 review

P1-1: Fail-closed entitlement gate (403 when no userId or lookup fails)
P1-2: Checkout requires auth (removed client-supplied userId fallback)
P1-3: Removed dual auth (PREMIUM_RPC_PATHS) from gateway, single entitlement path
P1-4: Typed features validator in cacheActions (v.object instead of v.any)
P1-5: Typed ConvexClient API ref (typeof api instead of Record<string,any>)
P1-6: Cache stampede mitigation via request coalescing (_inFlight map)
Round8-A: getUserId() returns Clerk user.id, hasUserIdentity() checks real identity
Round8-B: JWT verification pinned to algorithms: ['RS256']

- Updated entitlement tests for fail-closed behavior (7 tests pass)
- Removed userId arg from checkout client call
- Added env-aware Redis key prefix (live/test)
- Reduced cache TTL from 3600 to 900 seconds
- Added 5s timeout to Redis fetch calls

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(quick-5): resolve all P2 review issues from koala73 review

P2-1: dispute.lost now revokes entitlements (downgrades to free tier)
P2-2: rawPayload v.any() documented with JSDoc (intentional: external schema)
P2-3: Redis keys prefixed with live/test env (already in P1 commit)
P2-4: 5s timeout on Redis fetch calls (already in P1 commit)
P2-5: Cache TTL reduced from 3600 to 900 seconds (already in P1 commit)
P2-6: CONVEX_IS_DEV warning logged at module load time (once, not per-call)
P2-7: claimSubscription uses .first() instead of .unique() (race safety)
P2-8: toEpochMs fallback accepts eventTimestamp (all callers updated)
P2-9: hasUserIdentity() checks real identity (already in P1 commit)
P2-10: Duplicate JWT verification removed (already in P1 commit)
P2-11: Subscription queries bounded with .take(10)
P2-12: Entitlement subscription exposes destroyEntitlementSubscription()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(convex): resolve TS errors in http.ts after merge

Non-null assertions for anyApi dynamic module references and safe
array element access in timing-safe comparison.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(tests): update gateway tests for fail-closed entitlement system

Tests now reflect the new behavior where:
- API key + no auth session → 403 (entitlement check requires userId)
- Valid bearer + no entitlement data → 403 (fail-closed)
- Free bearer → 403 (entitlements unavailable)
- Invalid bearer → 401 (no session, forceKey kicks in)
- Public routes → 200 (unchanged)

The old tests asserted PREMIUM_RPC_PATHS + JWT role behavior which
was removed per koala73's P1-3 review (dual auth elimination).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(review): resolve 4 P1 + 2 P2 issues from koala73 round-9 review

P1 fixes:
- API-key holders bypass entitlement check (were getting 403)
- Browser checkout passes userId for identity bridge (ConvexClient
  has no setAuth yet, so createCheckout accepts optional userId arg)
- /pro pricing page embeds wm_user_id in Dodo checkout URL metadata
  so webhook can resolve identity for first-time purchasers
- Remove isProUser() gate from entitlement/billing init — all users
  now subscribe to entitlement changes so upgrades take effect
  immediately without manual page reload

P2 fixes:
- destroyEntitlementSubscription() called on teardown to clear stale
  premium state across SPA sessions (sign-out / identity change)
- Convex queries prefer resolveUserId(ctx) over client-supplied
  userId; documented as temporary until Clerk JWT wired into
  ConvexClient.setAuth()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(review): resolve 4 P1 + 1 P2 from koala73 round-10 review

P1 — Checkout identity no longer client-controlled:
  - createCheckout HMAC-signs userId with DODO_PAYMENTS_WEBHOOK_SECRET
  - Webhook resolveUserId only trusts metadata when HMAC signature is
    valid; unsigned/tampered metadata is rejected
  - /pro raw URLs no longer embed wm_user_id (eliminated URL tampering)
  - Purchases without signed metadata get synthetic "dodo:{customerId}"
    userId, claimable later via claimSubscription()

P1 — IDOR on Convex queries addressed:
  - Both getEntitlementsForUser and getSubscriptionForUser now reject
    mismatched userId when the caller IS authenticated (authedUserId !=
    args.userId → return defaults/null)
  - Created internal getEntitlementsByUserId for future gateway use
  - Pre-auth fallback to args.userId documented with TODO(clerk-auth)

P1 — Clerk identity bridge fixed:
  - user-identity.ts now uses getCurrentClerkUser() from clerk.ts
    instead of reading window.Clerk?.user (which was never assigned)
  - Signed-in Clerk users now correctly resolve to their Clerk user ID

P1 — Auth modal available for anonymous users:
  - Removed isProUser() gate from setupAuthWidget() in App.ts
  - Anonymous users can now click premium CTAs → sign-in modal opens

P2 — .take(10) subscription cap:
  - Bumped to .take(50) in both getSubscriptionForUser and
    getActiveSubscription to avoid missing active subscriptions

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(review): resolve remaining P2/P3 feedback from koala73 + greptile on PR #2024

JWKS: consolidate duplicate singletons — server/_shared/auth-session.ts now
imports the shared JWKS from server/auth-session.ts (eliminates redundant cold-start fetch).

Webhook: remove unreachable retry branch — webhookEvents.status is always
"processed" (inserted only after success, rolled back on throw). Dead else removed.

YAGNI: remove changePlan action + frontend stub (no callers — plan changes use
Customer Portal). Remove unused by_status index on subscriptions table.

DRY: consolidate identical pro_monthly/pro_annual into shared PRO_FEATURES constant.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): read CLERK_JWT_ISSUER_DOMAIN lazily in getJWKS() — fixes CI bearer token tests

The shared getJWKS() was reading the env var from a module-scope const,
which freezes at import time. Tests set the env var in before() hooks
after import, so getJWKS() returned null and bearer tokens were never
verified — causing 401 instead of 403 on entitlement checks.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(payments): resolve all blocking issues from review rounds 1-2

Data integrity:
- Use .first() instead of .unique() on entitlements.by_userId to survive
  concurrent webhook retries without permanently crashing reads
- Add typed paymentEventStatus union to schema; replace dynamic dispute
  status string construction with explicit lookup map
- Document accepted 15-min Redis cache sync staleness bound
- Document webhook endpoint URL in .env.example

Frontend lifecycle:
- Add listeners.clear() to destroyEntitlementSubscription (prevents
  stale closure reload loop on destroy/re-init cycles)
- Add destroyCheckoutOverlay() to reset initialized flag so new layouts
  can register their success callbacks
- Complete PanelLayoutManager.destroy() — add teardown for checkout
  overlay, payment failure banner, and entitlement change listener
- Preserve currentState across destroy; add resetEntitlementState()
  for explicit reset (e.g. logout) without clearing on every cycle

Code quality:
- Export DEV_USER_ID and isDev from lib/auth.ts; remove duplicates
  from subscriptionHelpers.ts (single source of truth)
- Remove DODO_PAYMENTS_API_KEY fallback from billing.ts
- Document why two Dodo SDK packages coexist (component vs REST)

* fix(payments): address round-3 review findings — HMAC key separation, auth wiring, shape guards

- Introduce DODO_IDENTITY_SIGNING_SECRET separate from webhook secret (todo 087)
  Rotating the webhook secret no longer silently breaks userId identity signing
- Wire claimSubscription to Clerk sign-in in App.ts (todo 088)
  Paying anonymous users now have their entitlements auto-migrated on first sign-in
- Promote getCustomerPortalUrl to public action + wire openBillingPortal (todo 089)
  Manage Billing button now opens personalized portal instead of generic URL
- Add rate limit on claimSubscription (todo 090)
- Add webhook rawPayload shape guard before handler dispatch (todo 096)
  Malformed payloads return 200 with log instead of crashing the handler
- Remove dead exports: resetEntitlementState, customerPortal wrapper (todo 091)
- Fix let payload; implicit any in webhookHandlers.ts (todo 092)
- Fix test: use internal.* for internalMutation seedProductPlans (todo 095)

Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>

* fix(payments): address round-4 P1 findings — input validation, anon-id cleanup

- claimSubscription: replace broken rate-limit (bypassed for new users,
  false positives on renewals) with UUID v4 format guard + self-claim
  guard; prevents cross-user subscription theft via localStorage injection
- App.ts: always remove wm-anon-id after non-throwing claim completion
  (not only when subscriptions > 0); adds optional chaining on result.claimed;
  prevents cold Convex init on every sign-in for non-purchasers

Resolves todos 097, 098, 099, 100

* fix(payments): address round-5 review findings — regex hoist, optional chaining

- Hoist ANON_ID_REGEX to module scope (was re-allocated on every call)
- Remove /i flag — crypto.randomUUID() always produces lowercase
- result.claimed accessed directly (non-optional) — mutation return is typed
- Revert removeItem from !client || !api branch — preserve anon-id on
  infrastructure failure; .catch path handles transient errors

* fix(billing): wire setAuth, rebind watches, revoke on dispute.lost, defer JWT, bound collect

- convex-client.ts: wire client.setAuth(getClerkToken) so claimSubscription and
  getCustomerPortalUrl no longer throw 'Authentication required' in production
- clerk.ts: expose clearClerkTokenCache() for force-refresh handling
- App.ts: rebind entitlement + subscription watches to real Clerk userId on every
  sign-in (destroy + reinit), fixing stale anon-UUID watches post-claim
- subscriptionHelpers.ts: revoke entitlement to 'free' on dispute.lost + sync Redis
  cache; previously only logged a warning leaving pro access intact after chargeback
- gateway.ts: compute isTierGated before resolveSessionUserId; defer JWKS+RS256
  verification to inside if (isTierGated) — eliminates JWT work on ~136 non-gated endpoints
- billing.ts: .take(1000) on paymentEvents collect — safety bound preventing runaway
  memory on pathological anonymous sessions before sign-in

Closes P1: setAuth never wired (claimSubscription always throws in prod)
Closes P2: watch rebind, dispute.lost revocation, gateway perf, unbounded collect

* fix(security): address P1 review findings from round-6 audit

- Remove _debug block from validateApiKey that contained all valid API keys
  in envVarRaw/parsedKeys fields (latent full-key disclosure risk)
- Replace {db: any} with QueryCtx in getEntitlementsHandler (Convex type safety)
- Add pre-insert re-check in upsertEntitlements with OCC race documentation
- Fix dispute.lost handler to use eventTimestamp instead of Date.now() for
  validUntil/updatedAt (preserves isNewerEvent out-of-order replay protection)
- Extract getFeaturesForPlan("free") to const in dispute.lost (3x → 1x call)

Closes todos #103, #106, #107, #108

* fix(payments): address round-6 open items — throw on shape guard, sign-out cleanup, stale TODOs

P2-4: Webhook shape guards now throw instead of returning silently,
so Dodo retries on malformed payloads instead of losing events.

P3-2: Sign-out branch now destroys entitlement and subscription
watches for the previous userId.

P3: Removed stale TODO(clerk-auth) comments — setAuth() is wired.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(payments): address codex review — preserve listeners, honest TODOs, retry comment

- destroyEntitlementSubscription/destroySubscriptionWatch no longer clear
  listeners — PanelLayout registers them once and they must survive auth
  transitions (sign-out → sign-in would lose the premium-unlock reload)
- Restore TODO(auth) on entitlements/billing public queries — the userId
  fallback is a real trust gap, not just a cold-start race
- Add comment on webhook shape guards acknowledging Dodo retry budget
  tradeoff vs silent event loss

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(types): restore non-null assertions in convex/http.ts after merge

Main removed ! and as-any casts, but our branch's generated types
make anyApi properties possibly undefined. Re-added to fix CI typecheck.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(types): resolve TS errors from rebase — cast internal refs, remove unused imports

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: regenerate package-lock.json — add missing uqr@0.1.2

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(payments): resolve P2/P3 review findings for PR #2024

- Bound and parallelize claimSubscription reads with Promise.all (4x queries
  -> single round trip; .collect() -> .take() to cap memory)
- Add returnUrl allowlist validation in createCheckout to prevent open redirect
- Make openBillingPortal return Promise<string | null> for agent-native callers
- Extend isCallerPremium with Dodo entitlement tier check (tier >= 1 is premium,
  unifying Clerk role:pro and Dodo subscriber as two signals for the same gate)
- Call resetEntitlementState() on sign-out to prevent entitlement state leakage
  across sessions (destroyEntitlementSubscription preserves state for reconnects;
  resetEntitlementState is the explicit sign-out nullifier)
- Merge handlePaymentEvent + handleRefundEvent -> handlePaymentOrRefundEvent
  (type inferred from event prefix; eliminates duplicate resolveUserId call)
- Remove _testCheckEntitlement DI export from entitlement-check.ts; inline
  _checkEntitlementCore into checkEntitlement; tests now mock getCachedJson
- Collapse 4 duplicate dispute status tests into test.each
- Fix stale entitlement variable name in claimSubscription return value

* fix(payments): harden auth and checkout ownership

* fix(gateway): tighten auth env handling

* fix(gateway): use convex site url fallback

* fix(app): avoid redundant checkout resume

* fix(convex): cast alertRules internal refs for PR-branch generated types

---------

Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
 2026-04-03 00:25:18 +04:00
Elie Habib
5ba7523629 chore: bump version to 2.6.6 (#2601) 
Forces new Sentry release tag so CSP listener fixes are tracked
separately from 2.6.5 sessions still running old code.
 2026-04-01 22:51:37 +04:00
Elie Habib
9b2c944eae fix(telegram): show copyable /start command for pairing (#2584) 
* fix(telegram): show /start command with copy button instead of unreliable deep link

Telegram desktop ignores ?start=TOKEN for already-started bots — the
deep link opens the conversation but sends /start without the payload.

Replace the "Open Telegram" button UI with a copyable /start TOKEN command
so users can paste it directly into the bot, which works on all clients.

The deep link on the bot username is kept as a convenience to open
Telegram, but pairing no longer depends on the start parameter being
forwarded.

* fix(telegram): add Open Telegram button alongside copy command

* feat(telegram): add QR code to pairing UI for mobile users

Generates an inline SVG QR code (uqr, no canvas required) encoding the
t.me deep link so mobile users can scan instead of copy/paste. Desktop
users still have the copy command + Open Telegram button.

Layout: QR on the left (88×88), copy row + Open Telegram on the right.

* fix(telegram): implement full layered pairing UX

- Open Telegram is now the primary action (button first)
- Copy /start command is the visible fallback, always shown
- QR code moves to right side as mobile convenience (not primary path)
- Polling never calls reloadNotifSection on expiry; instead renders an
  in-place expired state with Generate new code button so users don't
  lose context
- Generate new code and Try again re-use the same startTelegramPairing
  handler without backing out
- Countdown shows Waiting... Ns instead of bare Ns
- Clipboard fallback via execCommand when navigator.clipboard unavailable
- Loading state shows Generating code... while token is being created

* fix(telegram): fix clipboard fallback for unavailable Clipboard API

Two bugs from review:
- navigator.clipboard may be undefined; accessing .writeText on it threw
  synchronously before .catch could run
- document.execCommand('copy') without a selected element copies nothing

Fix: guard with navigator.clipboard?.writeText, and in the fallback create
a temporary off-screen textarea, select its content, then execCommand.
 2026-04-01 00:02:26 +04:00
Elie Habib
46d69547ad feat(analyst): suggest Widget Creator for visual/chart queries (#2546) 
* feat(analyst): suggest Widget Creator for visual/chart queries

When a user asks the Chat Analyst for something that could be visualised
(charts, price comparisons, dashboards), the backend detects that intent
via a keyword regex on the query and emits an SSE action event before
the first token. The frontend renders a "Create chart widget →" button
in the analyst bubble; clicking it opens the Widget Creator pre-filled
with the original query.

- api/chat-analyst.ts: VISUAL_INTENT_RE + buildActionEvents() +
  prependSseEvents() replaces prependSseEvent; action event emitted
  between meta and the first LLM delta
- ChatAnalystPanel.ts: ActionEvent interface, renderActionChip() method,
  action event handling in readStream()
- WidgetChatModal.ts: initialMessage? option pre-fills textarea
- panel-layout.ts: wm:open-widget-creator listener calls
  openWidgetChatModal with onComplete → addCustomWidget
- main.css: .chat-action-chip accent-coloured button style

🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.ai/claude-code) + Compound Engineering v2.49.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(analyst): chip reusable + extract action logic + add tests

P1: Remove { once: true } from action chip click handler so the
    Create chart widget button remains clickable after the modal
    is closed or fails preflight.

P2: Extract VISUAL_INTENT_RE and buildActionEvents into
    server/worldmonitor/intelligence/v1/chat-analyst-actions.ts
    (exported) and add 14 assertions to chat-analyst.test.mts:
    - visual queries emit suggest-widget action with correct fields
    - all five Quick Actions produce no chip
    - bare "chart" (UN Charter, "chart a course") is not matched
    - regex is case-insensitive

* fix(chat-analyst): extend VISUAL_INTENT_RE to match intermediate subject nouns

Add optional one-word gap in chart/graph/plot verb arms so natural queries
like 'chart oil prices vs gold' and 'graph interest rates' are detected.
Add rates? and performance to graph/plot arms for consistency.
Add 3 new test cases covering the intermediate-noun pattern.

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-03-30 17:01:21 +04:00
Elie Habib
a8a724ec81 fix(deps): patch node-forge, srvx, yaml via npm overrides (#2460) 
* fix(deps): add npm overrides for node-forge, srvx, yaml vulnerabilities

Pins transitive dependency patches via npm overrides to address 13 open
Dependabot alerts without waiting for upstream packages to update:

- node-forge 1.4.0: fixes 4 HIGH CVEs (cert chain bypass, Ed25519/RSA
  signature forgery, BigInteger DoS) — pulled via listhen → @nuxt/cli
- srvx 0.11.13: fixes MEDIUM middleware bypass via absolute URI
  — pulled via @nuxt/cli → @vercel/analytics
- yaml 2.8.3: fixes MEDIUM stack overflow via deeply nested YAML
  — pulled via vite, nuxt

picomatch excluded: v4 override would break micromatch@4 (v2 API).
brace-expansion excluded: 1.x→5.x is a major breaking jump.
Both are dev-only tools with no untrusted input attack surface.

* fix(deps): drop yaml override and use caret ranges for node-forge/srvx

Greptile flagged that "yaml": "2.8.3" was performing a cross-major
upgrade for cosmiconfig@7 (requires yaml@^1.10.0) — same risk we
explicitly avoided for picomatch and brace-expansion.

Removed yaml override entirely: vite/nuxt already resolve yaml@2.8.3
naturally via their ^2.x range, so the override was redundant. The
cosmiconfig nested yaml@1.10.3 is now left intact.

Switched node-forge and srvx from exact pins to caret ranges (^1.4.0,
^0.11.13) to match the existing override style and allow future patches.
 2026-03-29 08:35:33 +04:00
Elie Habib
f3b0280227 feat(economic): EIA weekly crude oil inventory seeder (#2142) (#2168) 
* feat(economic): EIA weekly crude oil inventory seeder (#2142)

- scripts/seed-economy.mjs: add fetchCrudeInventories() fetching WCRSTUS1, compute weeklyChangeMb, write economic:crude-inventories:v1 (10-day TTL)
- proto/worldmonitor/economic/v1/get_crude_inventories.proto: new proto with CrudeInventoryWeek and GetCrudeInventories RPC
- server/worldmonitor/economic/v1/get-crude-inventories.ts: RPC handler reading seeded key with getCachedJson(..., true)
- server/worldmonitor/economic/v1/handler.ts: wire in getCrudeInventories
- server/gateway.ts: add static cache tier for /api/economic/v1/get-crude-inventories
- api/health.js: crudeInventories in BOOTSTRAP_KEYS + SEED_META (maxStaleMin: 20160, 2x weekly cadence)
- src/services/economic/index.ts: add fetchCrudeInventoriesRpc() with circuit breaker
- src/components/EnergyComplexPanel.ts: surface 8-week sparkline and WoW change in energy panel
- src/app/data-loader.ts: call fetchCrudeInventoriesRpc() in loadOilAnalytics()

* fix: remove stray market-implications gateway entry from crude-inventories branch

* fix(crude-inventories): address ce-review P1/P2 findings before merge

- api/bootstrap.js: register crudeInventories in BOOTSTRAP_CACHE_KEYS + SLOW_KEYS (P1-001)
- server/_shared/cache-keys.ts: add crudeInventories key + tier to match bootstrap.js
- api/health.js: remove bundled marketImplications (belongs in separate PR) (P1-002)
- src/services/economic/index.ts: add isFeatureAvailable('energyEia') gate (P2-003)
- src/services/economic/index.ts: use getHydratedData('crudeInventories') on first load
- proto/get_crude_inventories.proto: weekly_change_mb → optional double (P2-004)
- scripts/seed-economy.mjs: CRUDE_INVENTORIES_TTL 10d → 21d (3× cadence) (P2-005)
- scripts/seed-economy.mjs: period format validation with YYYY-MM-DD regex (P3-007)
- src/app/data-loader.ts: warn on crude fetch rejection (P2-006)

* fix(crude-inventories): schema validation, MIN_ITEMS gate, handler logging, raw=true docs

- Handler: document raw=true param, log errors instead of silent catch
- Seeder: CRUDE_MIN_WEEKS=4 guard prevents quota-hit empty writes
- Seeder: isValidWeek() schema validation before Redis write

* chore: regenerate openapi docs after rebase (adds getCrudeInventories + getEconomicCalendar)

* fix(gateway): add list-market-implications to RPC_CACHE_TIER

* chore: exclude todos/ from markdownlint
 2026-03-27 09:42:26 +04:00
Elie Habib
a969a9e3a3 feat(auth): integrate clerk.dev (#1812) 
* feat(auth): integrate better-auth with @better-auth/infra dash plugin

Wire up better-auth server config with the dash() plugin from
@better-auth/infra, and the matching sentinelClient() on the
client side. Adds BETTER_AUTH_API_KEY to .env.example.

* feat(auth): swap @better-auth/infra for @convex-dev/better-auth

[10-01 task 1] Install @convex-dev/better-auth@0.11.2, remove
@better-auth/infra, delete old server/auth.ts skeleton, rewrite
auth-client.ts to use crossDomainClient + convexClient plugins.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): create Convex auth component files

[10-01 task 2] Add convex.config.ts (register betterAuth component),
auth.config.ts (JWT/JWKS provider), auth.ts (better-auth server with
Convex adapter, crossDomain + convex plugins), http.ts (mount auth
routes with CORS). Uses better-auth/minimal for lighter bundle.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): add admin, organization, and dash plugins

[10-01] Re-install @better-auth/infra for dash() plugin to enable
dash.better-auth.com admin dashboard. Add admin() and organization()
plugins from better-auth/plugins for user and org management.
Update both server (convex/auth.ts) and client (auth-client.ts).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): drop @better-auth/infra (Node.js deps incompatible with Convex V8)

Keep admin() and organization() from better-auth/plugins (V8-safe).
@better-auth/infra's dash() transitively imports SAML/SSO with
node:crypto, fs, zlib — can't run in Convex's serverless runtime.
Dashboard features available via admin plugin endpoints instead.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(11-01): create auth-state.ts with OTT handler and session subscription

- Add initAuthState() for OAuth one-time token verification on page load
- Add subscribeAuthState() reactive wrapper around useSession nanostore atom
- Add getAuthState() synchronous snapshot getter
- Export AuthUser and AuthSession types for UI consumption

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(11-01): add Google OAuth provider and wire initAuthState into App.ts

- Add socialProviders.google with GOOGLE_CLIENT_ID/SECRET to convex/auth.ts
- Add all variant subdomains to trustedOrigins for cross-subdomain CORS
- Call initAuthState() in App.init() before panelLayout.init()
- Add authModal field to AppContext interface (prepares for Plan 02)
- Add authModal: null to App constructor state initialization

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(11-02): create AuthModal with Sign In/Sign Up tabs and Google OAuth

- Sign In tab: email/password form calling authClient.signIn.email()
- Sign Up tab: name/email/password form calling authClient.signUp.email()
- Google OAuth button calling authClient.signIn.social({ provider: 'google', callbackURL: '/' })
- Auto-close on successful auth via subscribeAuthState() subscription
- Escape key, overlay click, and X button close the modal
- Loading states, error display, and client-side validation

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(11-02): add AuthHeaderWidget, mount in header, add auth CSS

- AuthHeaderWidget: reactive header widget showing Sign In button (anonymous) or avatar + dropdown (authenticated)
- User dropdown: name, email, Free tier badge, Sign Out button calling authClient.signOut()
- setupAuthWidget() in EventHandlerManager creates modal + widget, mounts at authWidgetMount span
- authWidgetMount added to panel-layout.ts header-right, positioned before download wrapper
- setupAuthWidget() called from App.ts after setupUnifiedSettings()
- Full auth CSS: modal styles, tabs, forms, Google button, header widget, avatar, dropdown

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(11-02): add localhost:3000 to trustedOrigins for local dev CORS

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): remove admin/organization plugins that break Convex adapter validator

The admin() plugin adds banned/role fields to user creation data, but the
@convex-dev/better-auth adapter validator doesn't include them. These plugins
are Phase 12 work — will re-add with additionalFields config when needed.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-01): add Resend email transport, verification + reset callbacks, role field

- Install resend SDK for transactional email
- Add emailVerification with sendOnSignUp:true and fire-and-forget Resend callbacks
- Add sendResetPassword callback with 1-hour token expiry
- Add user.additionalFields.role (free/pro, input:false, defaultValue:free)
- Create userRoles fallback table in schema with by_userId index
- Create getUserRole query and setUserRole mutation in convex/userRoles.ts
- Lazy-init Resend client to avoid Convex module analysis error

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-01): enhance auth-state with emailVerified and role fields

- Add emailVerified (boolean) and role ('free' | 'pro') to AuthUser interface
- Fetch role from Convex userRoles table via HTTP query after session hydration
- Cache role per userId to avoid redundant fetches
- Re-notify subscribers asynchronously when role is fetched for a new user
- Map emailVerified from core better-auth user field (default false)
- Derive Convex cloud URL from VITE_CONVEX_SITE_URL env var

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(12-01): add Convex generated files from deployment

- Track convex/_generated/ files produced by npx convex dev --once

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-03): create panel-gating service with auth-aware showGatedCta

- Add PanelGateReason enum (NONE/ANONYMOUS/UNVERIFIED/FREE_TIER)
- Add getPanelGateReason() computing gating from AuthSession + premium flag
- Add Panel.showGatedCta() rendering auth-aware CTA overlays
- Add Panel.unlockPanel() to reverse locked state
- Extract lockSvg to module-level const shared by showLocked/showGatedCta
- Add i18n keys: signInToUnlock, signIn, verifyEmailToUnlock, resendVerification, upgradeDesc, upgradeToPro

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-02): add forgot password flow, password reset form, and token detection

- Widen authModal interface in app-context.ts to support reset-password mode and setResetToken
- AuthModal refactored with 4 views: signin, signup, forgot-password, reset-password
- Forgot password view sends reset email via authClient.requestPasswordReset
- Reset password form validates matching passwords and calls authClient.resetPassword
- auth-state.ts detects ?token= param from email links, stores as pendingResetToken
- App.ts routes pending reset token to auth modal after UI initialization
- CSS for forgot-link, back-link, and success message elements

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-02): add email verification banner to AuthHeaderWidget and tier badge

- Show non-blocking verification banner below header for unverified users
- Banner has "Resend" button calling authClient.sendVerificationEmail
- Banner is dismissible (stored in sessionStorage, reappears next session)
- Tier badge dynamically shows Free/Pro based on user.role
- Pro badge has gradient styling distinct from Free badge
- Dropdown shows unverified status indicator with yellow dot
- Banner uses fixed positioning, does not push content down
- CSS for banner, pro badge, and verification status indicators

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(12-03): wire reactive auth-based gating into panel-layout

- Add WEB_PREMIUM_PANELS Set (stock-analysis, stock-backtest, daily-market-brief)
- Subscribe to auth state changes in PanelLayoutManager.init()
- Add updatePanelGating() iterating panels with getPanelGateReason()
- Add getGateAction() returning CTA callbacks per gate reason
- Remove inline showLocked() calls for web premium panels
- Preserve desktop _lockPanels for forecast, oref-sirens, telegram-intel
- Clean up auth subscription in destroy()

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(13-01): create auth-token utility and inject Bearer header in web fetch redirect

- Add src/services/auth-token.ts with getSessionBearerToken() that reads session token from localStorage
- Add WEB_PREMIUM_API_PATHS Set for the 4 premium market API paths
- Inject Authorization: Bearer header in installWebApiRedirect() for premium paths when session exists
- Desktop installRuntimeFetchPatch() left unchanged (API key only)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(13-01): create server-side session validation module

- Add server/auth-session.ts with validateBearerToken() for Vercel edge gateway
- Validates tokens via Convex /api/auth/get-session with Better-Auth-Cookie header
- Falls back to userRoles:getUserRole Convex query for role resolution
- In-memory cache with 60s TTL and 100-entry cap
- Network errors not cached to allow retry on next request

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(13-02): add bearer token fallback auth for premium API endpoints

- Dynamic import of auth-session.ts when premium endpoint + API key fails
- Valid pro session tokens fall through to route handler
- Non-pro authenticated users get 403 'Pro subscription required'
- Invalid/expired tokens get 401 'Invalid or expired session'
- Non-premium endpoints and static API key flow unchanged

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): sign-in button invisible in dark theme — white on white

--accent is #fff in dark theme, so background: var(--accent) + color: #fff
was invisible. Changed to transparent background with var(--text) color.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): add premium panel keys to full and finance variant configs

stock-analysis, stock-backtest, and daily-market-brief were defined in
the shared panels.ts but missing from variant DEFAULT_PANELS, causing
shouldCreatePanel() to return false and panel gating CTAs to never render.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* test(auth): add Playwright smoke tests for auth UI (phases 12-13)

6 tests covering: Sign In button visibility, auth modal opening,
modal views (Sign In/Sign Up/Forgot Password), premium panel gating
for anonymous users, and auth token absence when logged out.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): remove role additionalField that breaks Convex component validator

The betterAuth Convex component has a strict input validator for the
user model that doesn't include custom fields. The role additionalField
caused ArgumentValidationError on sign-up. Roles are already stored in
the separate userRoles table — no data loss.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): use Authorization Bearer header for Convex session validation

Better-Auth-Cookie header returned null — the crossDomain plugin's
get-session endpoint expects Authorization: Bearer format instead.
Confirmed via curl against live Convex deployment.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): use verified worldmonitor.app domain for auth emails

Was using noreply@resend.dev (testing domain) which can't send to
external recipients. Switched to noreply@worldmonitor.app matching
existing waitlist/contact emails.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): await Resend email sends — Convex kills dangling promises

void (fire-and-forget) causes Convex to terminate the fetch before
Resend receives it. Await ensures emails actually get sent.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: update Convex generated auth files after config changes

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): guard against undefined VITE_CONVEX_SITE_URL in auth-state

The Convex cloud URL derivation crashed the entire app when
VITE_CONVEX_SITE_URL wasn't set in the build environment (Vercel
preview). Now gracefully defaults to empty string and skips role
fetching when the URL is unavailable.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(auth): add dash + organization plugins, remove Google OAuth, fix dark mode button

- Add @better-auth/infra dash plugin for hosted admin dashboard
- Add organization plugin for org management in dashboard
- Add dash.better-auth.com to trustedOrigins
- Remove Google OAuth (socialProviders, button, divider, CSS)
- Fix auth submit button invisible in dark mode (var(--accent) → #3b82f6)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): replace dash plugin with admin — @better-auth/infra incompatible with Convex V8

@better-auth/infra imports SSO/SAML libraries requiring Node.js built-ins
(crypto, fs, stream) which Convex's V8 runtime doesn't support.
Replaced with admin plugin from better-auth/plugins which provides
user management endpoints (set-role, list-users, ban, etc.) natively.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore: remove stale Convex generated files after plugin update

Convex dev regenerated _generated/ — the per-module JS files
(auth.js, http.js, schema.js, etc.) are no longer emitted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* chore(auth): remove organization plugin — will add in subsequent PR

Organization support (team accounts, invitations, member management)
is not wired into any frontend flow yet. Removing to keep the auth
PR focused on email/password + admin endpoints. Will add back when
building the org/team feature.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* docs: add authentication & panel gating guide

Documents the auth stack, panel gating configuration, server-side
session enforcement, environment variables, and user roles.
Includes step-by-step guide for adding new premium panels.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(test): stub panel-gating in RuntimeConfigPanel test harness

Panel.ts now imports @/services/panel-gating, which wasn't stubbed —
causing the real runtime.ts (with window.location) to be bundled,
breaking Node.js tests with "ReferenceError: location is not defined".

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): allow Vercel preview origins in Convex trustedOrigins

* fix(auth): broaden Convex trustedOrigins to cover *.worldmonitor.app previews

* fix(auth): use hostonly wildcard pattern for *.worldmonitor.app in trustedOrigins

* fix(auth): add Convex site origins to trustedOrigins

* fix(ci): add convex/ to vercel-ignore watched paths

* fix(auth): remove admin() plugin — adds banned/role fields rejected by Convex validator

* fix(auth): remove admin() plugin — injects banned/role fields rejected by Convex betterAuth validator

* feat(auth): replace email/password with email OTP passwordless flow

- Replace emailAndPassword + emailVerification with emailOTP plugin
- Rewrite AuthModal: email entry -> OTP code verification (no passwords)
- Remove admin() plugin (caused Convex schema validation errors)
- Remove email verification banner and UNVERIFIED gate reason (OTP
  inherently verifies email)
- Remove password reset flow (forgot/reset password views, token handling)
- Clean up unused CSS (tabs, verification banner, success messages)
- Update docs to reflect new passwordless auth stack

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(quick-2): harden Convex userRoles and add role cache TTL

- P0: Convert setUserRole from mutation to internalMutation (not callable from client)
- P2: Add 5-minute TTL to role cache in auth-state.ts
- P2: Add localStorage shape warning on auth-token.ts
- P3: Document getUserRole public query trade-off
- P3: Fix misleading cache comment in auth-session.ts

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(quick-2): auth widget teardown, E2E test rewrite, gateway comment

- P2: Store authHeaderWidget on AppContext, destroy in EventHandlerManager.destroy()
- P2: Also destroy authModal in destroy() to prevent leaked subscriptions
- P1: Rewrite E2E tests for 2-view OTP modal (email input + submit button)
- P1: Remove stale "Sign Up" and "Forgot Password" test assertions
- P2: Replace flaky waitForTimeout(5000) with Playwright auto-retry assertion
- P3: Add clarifying comment on premium bearer-token fallback in gateway

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(header): restructure header/footer, add profile editing, pro-gate playback/export

- Remove version, @eliehabib, GitHub link, and download button from header
- Move version + @eliehabib credit to footer brand line; download link to footer nav
- Move auth widget (profile avatar) to far right of header (after settings gear)
- Add default generic SVG avatar for users with no image and no name
- Add profile editing in auth dropdown: display name + avatar URL with Save/Cancel
- Add Settings shortcut in auth dropdown (opens UnifiedSettings)
- Gate Historical Playback and Export controls behind pro role (hidden for free users)
- Reactive pro-gate: subscribes to auth state changes, stores unsub in proGateUnsubscribers[]
- Clean up proGateUnsubscribers on EventHandlerManager.destroy() to prevent leaks
- Fix: render Settings button unconditionally (hidden via style), stable DOM structure
- Fix: typed updateUser call with runtime existence check instead of (any) cast
- Make initFooterDownload() private to match class conventions

* feat(analytics): add Umami auth integration and event tracking

- Wire analytics.ts facade to Umami (port from main #1914):
  search, country, map layers, panels, LLM, theme, language,
  variant switch, webcam, download, findings, deeplinks
- Add Window.umami shim to vite-env.d.ts
- Add initAuthAnalytics() that subscribes to auth state and calls
  identifyUser(id, role) / clearIdentity() on sign-in/sign-out
- Add trackSignIn, trackSignUp, trackSignOut, trackGateHit exports
- Call initAuthAnalytics() from App.ts after initAuthState()
- Track sign-in/sign-up (via isNewUser flag) in AuthModal OTP verify
- Track sign-out in AuthHeaderWidget before authClient.signOut()
- Track gate-hit for export, playback (event-handlers) and pro-banner

* feat(auth): professional avatar widget with colored initials and clean profile edit

- Replace white-circle avatar with deterministic colored initials (Gmail/Linear style)
- Avatar color derived from email hash across 8-color palette
- Dropdown redesigned: row layout with large avatar + name/email/tier info
- Profile edit form: name-only (removed avatar URL field)
- Remove Settings button from dropdown (gear icon in header is sufficient)
- Discord community widget: single CTA link, no redundant text label
- Add all missing CSS for dropdown interior, profile edit form, menu items

* fix(auth): lock down billing tier visibility and fix TOCTOU race

P1: getUserRole converted to internalQuery — billing tier no longer
accessible via any public Convex client API. Exposed only through
the new authenticated /api/user-role HTTP action which validates
the session Bearer token before returning the role.

P1: subscribeAuthState generation counter + AbortController prevents
rapid sign-in/sign-out from delivering stale role for wrong user.

P2: typed RawSessionUser/RawSessionValue interfaces replace any casts
at the better-auth nanostore boundary. fetchUserRole drops userId
param — server derives identity from Bearer token only.

P2: isNewUser heuristic removed from OTP verify — better-auth emailOTP
has no reliable isNewUser signal. All verifications tracked as
trackSignIn. OTP resend gets 30s client-side cooldown.

P2: auth-token.ts version pin comment added (better-auth@1.5.5 +
@convex-dev/better-auth@0.11.2). Gateway inner PREMIUM_RPC_PATHS
comment clarified to explain why it is not redundant.

Adds tests/auth-session.test.mts: 11 tests covering role fallback
endpoint selection, fail-closed behavior, and CORS origin matching.

* feat(quick-4): replace better-auth with Clerk JS -- packages, Convex config, browser auth layer

- Remove better-auth, @convex-dev/better-auth, @better-auth/infra, resend from dependencies
- Add @clerk/clerk-js and jose to dependencies
- Rewrite convex/auth.config.ts for Clerk issuer domain
- Simplify convex/convex.config.ts (remove betterAuth component)
- Delete convex/auth.ts, convex/http.ts, convex/userRoles.ts
- Remove userRoles table from convex/schema.ts
- Create src/services/clerk.ts with Clerk JS init, sign-in, sign-out, token, user metadata, UserButton
- Rewrite src/services/auth-state.ts backed by Clerk (same AuthUser/AuthSession interface)
- Delete src/services/auth-client.ts (better-auth client)
- Delete src/services/auth-token.ts (localStorage token scraping)
- Update .env.example with Clerk env vars, remove BETTER_AUTH_API_KEY

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* feat(quick-4): UI components, runtime fetch, server-side JWT, CSP, and tests

- Delete AuthModal.ts, create AuthLauncher.ts (thin Clerk.openSignIn wrapper)
- Rewrite AuthHeaderWidget.ts to use Clerk UserButton + openSignIn
- Update event-handlers.ts to use AuthLauncher instead of AuthModal
- Rewrite runtime.ts enrichInitForPremium to use async getClerkToken()
- Rewrite server/auth-session.ts for jose-based JWT verification with cached JWKS
- Update vercel.json CSP: add *.clerk.accounts.dev to script-src and frame-src
- Add Clerk CSP tests to deploy-config.test.mjs
- Rewrite e2e/auth-ui.spec.ts for Clerk UI
- Rewrite auth-session.test.mts for jose-based validation
- Use dynamic import for @clerk/clerk-js to avoid Node.js test breakage

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): allow Clerk Pro users to load premium data on web

The data-loader gated premium panel loading (stock-analysis, stock-backtest,
daily-market-brief) on WORLDMONITOR_API_KEY only, which is desktop-only.
Web users with Clerk Pro auth were seeing unlocked panels stuck on "Loading..."
because the requests were never made.

Added hasPremiumAccess() helper that checks for EITHER desktop API key OR
Clerk Pro role, matching the migration plan Phase 7 requirements.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): address PR #1812 review — all 4 merge blockers + 3 gaps

Blockers:
1. Remove stale Convex artifacts (http.js, userRoles.js, betterAuth
   component) from convex/_generated/api.d.ts
2. isProUser() now checks getAuthState().user?.role === 'pro' alongside
   legacy localStorage keys
3. Finance premium refresh scheduling now fires for Clerk Pro web users
   (not just API key holders)
4. JWT verification now validates audience: 'convex' to reject tokens
   scoped to other Clerk templates

Gaps:
5. auth-session tests: 10 new cases (valid pro/free, expired, wrong
   key/audience/issuer, missing sub/plan, JWKS reuse) using self-signed
   keys + local JWKS server
6. premium-stock-gateway tests: 4 new bearer token cases (pro→200,
   free→403, invalid→401, public unaffected)
7. docs/authentication.mdx rewritten for Clerk (removed all better-auth
   references, updated stack/files/env vars/roles sections)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: address P1 reactive Pro UI + P2 daily-market-brief + P3 stale env vars

P1 — In-session Pro UI changes no longer require a full reload:
- setupExportPanel: removed early isProUser() return, always creates
  and relies on reactive subscribeAuthState show/hide
- setupPlaybackControl: same pattern — always creates, reactive gate
- Custom widget panels: always loaded regardless of Pro status
- Pro add-panel and MCP add-panel blocks: always rendered, shown/hidden
  reactively via subscribeAuthState callback
- Flight search wiring: always wired, checks Pro status inside callback
  so mid-session sign-ins work immediately

P2 — daily-market-brief added to hasPremiumAccess() block in loadAllData()
so Clerk Pro web users get initial data load (was only primed in
primeVisiblePanelData, missing from the general reload path)

P3 — Removed stale CONVEX_SITE_URL and VITE_CONVEX_SITE_URL from
docs/authentication.mdx env vars table (neither is referenced in codebase)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix: add isProUser import, populate PREMIUM_RPC_PATHS, and fix bearer token auth flow

- Added missing isProUser import in App.ts (fixes typecheck)
- Populated PREMIUM_RPC_PATHS with stock analysis endpoints
- Restructured gateway auth: trusted browser origins bypass API key for
  premium endpoints (client-side isProUser gate), while bearer token
  validation runs as a separate step for premium paths when present

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(gateway): require credentials for premium paths + defer free-tier enforcement until auth ready

P0: Removed trusted-origin bypass for premium endpoints — Origin header
is spoofable and cannot be a security boundary. Premium paths now always
require either an API key or valid bearer token.

P1: Deferred panel/source free-tier enforcement until auth state resolves.
Previously ran in the constructor before initAuthState(), causing Clerk Pro
users to have their panels/sources trimmed on every startup.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

* fix(auth): apply WorldMonitor design system to Clerk modal

Theme-aware appearance config passed to clerk.load(), openSignIn(),
and mountUserButton(). Dark mode: dark bg (#111), green primary
(#44ff88), monospace font. Light mode: white bg, green-600 primary
(#16a34a). Reads document.documentElement.dataset.theme at call time
so theme switches are respected.

* fix(auth): gate Clerk init and auth widget behind BETA_MODE

Clerk auth initialization and the Sign In header widget are now only
activated when localStorage `worldmonitor-beta-mode` is set to "true",
allowing silent deployment for internal testing before public rollout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(auth): gate Clerk init and auth widget behind isProUser()

Clerk auth initialization and the Sign In header widget are now only
activated when the user has wm-widget-key or wm-pro-key in localStorage
(i.e. isProUser() returns true), allowing silent deployment for internal
testing before public rollout.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(data-loader): replace stale isProUser() with hasPremiumAccess()

loadMarketImplications() still referenced the removed isProUser import,
causing a TS2304 build error. Align with the rest of data-loader.ts
which uses hasPremiumAccess() (checks both API key and Clerk auth).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(auth): address PR #1812 review — P1 security fixes + P2 improvements

P1 fixes:
- Add algorithms: ['RS256'] allowlist to jwtVerify (prevents alg:none bypass)
- Reset loadPromise on Clerk init failure (allows retry instead of permanent breakage)

P2 fixes:
- Extract PREMIUM_RPC_PATHS to shared module (eliminates server/client divergence risk)
- Add fail-fast guard in convex/auth.config.ts for missing CLERK_JWT_ISSUER_DOMAIN
- Add 50s token cache with in-flight dedup to getClerkToken() (prevents concurrent races)
- Sync Clerk CSP entries to index.html and tauri.conf.json (previously only in vercel.json)
- Type clerkInstance as Clerk instead of any

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(auth): clear cached token on signOut()

Prevents stale token from being returned during the ≤50s cache window
after a user signs out.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Sebastien Melki <sebastien@anghami.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Sebastien Melki <sebastienmelki@gmail.com>
 2026-03-26 13:47:22 +02:00
Elie Habib
b24c1374b7 fix(live-news): use hls.js for HLS playback in Chrome/Firefox (#1998) 
* fix(live-news): use hls.js for HLS playback in Chrome/Firefox

Chrome and Firefox do not support HLS natively in <video>, so setting
video.src to an .m3u8 URL fails immediately, burns the per-channel
HLS cooldown, and causes a fallback to YouTube which returns error 150
for channels like CNN that prohibit embedding.

Now renderNativeHlsPlayer detects native support via canPlayType and
uses hls.js for non-Safari browsers. Only fatal hls.js errors trigger
the cooldown and YouTube fallback path.

* fix(live-news): lazy-load hls.js and add media element error fallback

P1: Add video.addEventListener('error') in the hls.js branch so decode
failures that surface on the HTMLMediaElement (not through a fatal
hls.js event) still trigger the cooldown and YouTube fallback. A
once-guard prevents double-firing when both listeners fire together.

P2: Remove top-level import Hls from 'hls.js' and replace with a
dynamic import() inside renderNativeHlsPlayer. hls.js is now only
fetched when canPlayType returns falsy, so Safari and non-Live-News
users pay no bundle cost.

Also guard the post-await path against channel switches that occur
while the hls.js chunk is in flight.
 2026-03-21 17:49:03 +04:00
DrDavidL
7fdfea854b security: add unicode safety guard to hooks and CI (#1710) 
* security: add unicode safety guard to hooks and CI

* fix(unicode-safety): drop FE0F, PUA; fix col tracking; scan .husky/

- Remove FE0F (emoji presentation selector) from suspicious set — it
  false-positives on ASCII keycap sequences (#️⃣ etc.) in source strings
- Remove Private Use Area (E000–F8FF) check — not a parser attack vector
  and legitimately used by icon font string literals
- Fix column tracking for astral-plane characters (cp > 0xFFFF): increment
  by 2 to match UTF-16 editor column positions
- Remove now-unused prevCp variable
- Add .husky/ to SCAN_ROOTS and '' to INCLUDED_EXTENSIONS so extensionless
  hook scripts (pre-commit, pre-push) are included in full-repo scans

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
 2026-03-19 08:48:08 +04:00
Elie Habib
c332072723 fix(widget): add @anthropic-ai/sdk to root deps for Railway (#1757) 
The relay dynamically imports @anthropic-ai/sdk but it was only in
scripts/package.json. Railway Nixpacks only installs root deps, so the
package was missing at runtime (ERR_MODULE_NOT_FOUND).
 2026-03-17 13:07:14 +04:00
Elie Habib
c6f5d6a8f1 fix(health): remove stale SEED_META for RPC-populated keys, bump to v2.6.5 (#1669) 
riskScores and serviceStatuses have data but permanently stale seed-meta
(no longer written by cachedFetchJson after PR #1649). ON_DEMAND_KEYS
only affects EMPTY status, not STALE_SEED. Removing their SEED_META
entries so health doesn't check freshness for keys that can't update it.

Also bumps version to 2.6.5.
 2026-03-15 19:27:18 +04:00
Elie Habib
39931456a1 feat(forecast): add structured scenario pipeline and trace export (#1646) 
* feat(forecast): add AI Forecasts prediction module (Pro-tier)

MiroFish-inspired prediction engine that generates structured forecasts
across 6 domains (conflict, market, supply chain, political, military,
infrastructure) using existing WorldMonitor data streams.

- Proto definitions for ForecastService with GetForecasts RPC
- Dedicated seed script (seed-forecasts.mjs) with 6 domain detectors,
  cross-domain cascade resolver, prediction market calibration, and
  trend detection via prior snapshot comparison
- Premium-gated RPC handler (PREMIUM_RPC_PATHS enforcement)
- Lazy-loaded ForecastPanel with domain filters, probability bars,
  trend arrows, signal evidence, and cascade links
- Health monitoring integration (seed-meta freshness tracking)
- Refresh scheduler with API key guard

* test(forecast): add 47 unit tests for forecast detectors and utilities

Covers forecastId, normalize, resolveCascades, calibrateWithMarkets,
computeTrends, and smoke tests for all 6 domain detectors. Exports
testable functions from seed script with direct-run guard.

* fix(forecast): domain mismatch 'infra' vs 'infrastructure', add panel category

- Seed script used 'infra' but ForecastPanel filtered on 'infrastructure',
  causing Infra tab to show zero results
- Added 'forecast' to intelligence category in PANEL_CATEGORY_MAP

* fix(forecast): move CSS to one-time injection, improve type safety

- P2: Move style block from setContent to one-time document.head injection
  to prevent CSS accumulation on repeated renders
- P3: Replace +toFixed(3) with Math.round for readability in seed script
- P3: Use Forecast type instead of any[] in RPC handler filter

* fix(forecast): handle sebuf proto data shapes from Redis

Detectors now normalize CII scores from server-side proto format
(combinedScore, TREND_DIRECTION_RISING, region) to uniform shape.
Outage severity handles proto enum format (SEVERITY_LEVEL_HIGH).
Added confidence floor of 0.3 for single-source predictions.

Verified against live Redis: 2 predictions generated (Iran infra
shutdown, IL political instability).

* feat(forecast): unlock AI Forecasts on web, lock desktop only (trial)

- Remove forecast RPC from PREMIUM_RPC_PATHS (web access is free)
- Panel locked on desktop only (same as oref-sirens/telegram-intel)
- Remove API key guards from data-loader and refresh scheduler
- Web users get full access during trial period

* chore: regenerate proto types with make generate

Re-ran make generate after rebasing on main. Plugin v0.7.0 dropped
@ts-nocheck from output, added it back to all 50 generated files.
Fixed 4 type errors from proto codegen changes:
- MarketSource enum -> string union type
- TemporalAnomalyProto -> TemporalAnomaly rename
- webcam lastUpdated number -> string

* chore: add proto freshness check to pre-push hook

Runs make generate before push and compares checksums of generated files.
If proto types are stale, blocks push with instructions to regenerate.
Skips gracefully if buf CLI is not installed.

* fix(forecast): use chokepoints v4 key, include ciiContribution in unrest

- P1: Switch chokepoints input from stale v2 to active v4 Redis key,
  matching bootstrap.js and cache-keys.ts
- P2: Add ciiContribution to unrest component fallback chain in
  normalizeCiiEntry so political detector reads the correct sebuf field

* feat(forecast): Phase 2 LLM scenario enrichment + confidence model

MiroFish-inspired enhancements:
- LLM scenario narratives via Groq/OpenRouter (narrative-only, no numeric
  adjustment). Evidence-grounded prompts with mandatory signal citation
  and few-shot examples from MiroFish's SECTION_SYSTEM_PROMPT_TEMPLATE.
- Top-4 predictions batched into single LLM call for cost efficiency.
- News context from newsInsights attached to all predictions for LLM
  prompt grounding (NOT in signals, cannot affect confidence).
- Deterministic confidence model: source diversity via SIGNAL_TO_SOURCE
  mapping (deduplicates cii+cii_delta, theater+indicators) + calibration
  agreement from prediction market drift. Floor 0.2, ceiling 1.0.
- Output validation: rejects scenarios without signal references.
- Truncated JSON repair for small model output.
- Structured JSON logging for LLM calls.
- Redis cache for LLM scenarios (1h TTL).
- 23 new tests (70 total), all passing.
- Live-tested: OpenRouter gemini-2.5-flash produces evidence-grounded
  scenario narratives from real WorldMonitor data.

* feat(forecast): Phase 3 multi-perspective scenarios, projections, data-driven cascades

MiroFish-inspired enhancements:
- Multi-perspective LLM analysis: top-2 predictions get strategic,
  regional, and contrarian viewpoints via combined LLM call
- Probability projections: domain-specific decay curves (h24/d7/d30)
  anchored to timeHorizon so probability equals projections[timeHorizon]
- Data-driven cascade rules: moved from hardcoded array to JSON config
  (scripts/data/cascade-rules.json) with schema validation, named
  predicate evaluators, unknown key rejection, and fallback to defaults
- 4 new cascade paths: infrastructure->supply_chain, infrastructure->market
  (both requiresSeverity:total), conflict->political, political->market
- Proto: added Perspectives and Projections messages to Forecast
- ForecastPanel: renders projections row and conditional perspectives toggle
- 89 tests (19 new), all passing
- Live-tested: OpenRouter produces perspectives from real data

* feat(forecast): Phase 4 data utilization + entity graph

Fixes data gaps that prevented 4 of 6 detectors from firing:
- Input normalizers: chokepoint v4 shape + GPS hexes-to-zones mapping
- Chokepoint warm-ping (production-only, requires WM_API_BASE_URL)
- Lowered CII conflict threshold from 70 to 60, gated on level=high|critical

4 new standalone detectors:
- UCDP conflict zones (10+ events per country)
- Cyber threat concentration (5+ threats per country)
- GPS jamming in maritime shipping zones (5 regions)
- Prediction markets as signals (60-90% probability markets)

Entity-relationship graph (file-based, 38 nodes):
- Countries, theaters, commodities, chokepoints, alliances
- Alias table resolves both ISO codes and display names
- Graph cascade discovery links predictions across entities

Result: 51 predictions (up from 1-2), spanning conflict, infrastructure,
and supply chain domains. 112 tests, all passing.

* fix(forecast): redis cache format, signal source mapping, type safety

Fresh-eyes audit fixes:
- BUG: redisSet used wrong Upstash API format (POST body with {value,ex}
  instead of command array ['SET',key,value,'EX',ttl]). LLM cache writes
  were silently failing, causing fresh LLM calls every run.
- BUG: prediction_market signal type missing from SIGNAL_TO_SOURCE,
  inflating confidence for market-derived predictions.
- CLEANUP: Remove unnecessary (f as any) casts in ForecastPanel since
  generated Forecast type already has projections/perspectives fields.
- CLEANUP: Bump health maxStaleMin from 60 to 90 to avoid false STALE
  alerts when LLM calls add latency to seed runs.

* feat(forecast): headline-entity matching with news corroboration signals

Uses entity graph aliases to match headlines to predictions by
country/theater (excludes commodity/infrastructure nodes to prevent
false positives). Predictions with matching headlines get a
news_corroboration signal visible in the panel.

Also fixes buildUserPrompt to merge unique headlines from ALL
predictions in the LLM batch (was only reading preds[0].newsContext).

Live-tested: 13 of 51 predictions now have corroborating headlines
(Iran, Israel, Syria, Ukraine, etc). 116 tests, all passing.

* feat(forecast): add country-codes.json for headline-entity matching

56 countries with ISO codes, full names, and scoring keywords (extracted
from src/config/countries.ts + UCDP-relevant additions). Used by
attachNewsContext for richer headline matching via getSearchTermsForRegion
which combines country-codes + entity graph + keyword aliases.

14/57 predictions now have news corroboration (limited by headline
coverage, not matching quality: only 8 headlines currently available).

* feat(forecast): read 300 headlines from news digest instead of 8

Read news:digest:v1:full:en (300 headlines across 16 categories) instead
of just news:insights:v1 topStories (8 headlines). Fallback to topStories
if digest is unavailable.

Result: news corroboration jumped from 25% to 64% (38/59 predictions).

* fix(forecast): handle parenthetical country names in headline matching

Strip suffixes like '(Zaire)', '(Burma)', '(Soviet Union)' from UCDP
region names before matching against country-codes.json. Also use
includes() for reverse name lookup to catch partial matches.

Corroboration: 64% -> 69% (41/59). Remaining 18 unmatched are countries
with no current English-language news coverage.

* fix(forecast): cache validated LLM output, add digest test, log cache errors

Fresh-eyes audit fixes:
- Combined LLM cache now stores only validated items (was caching raw
  unvalidated output, serving potentially invalid scenarios on cache hit)
- redisSet logs warnings on failure (was silently swallowing all errors)
- Added digest-based test for attachNewsContext (primary path was untested)
- Fixed test arity: attachNewsContext(preds, news, digest) with 3 params

* fix(forecast): remove dead confidenceFromSources, reduce warm-ping timeout

- P2: Remove confidenceFromSources (dead code, computeConfidence overwrites
  all initial confidence values). Inline the formula in original detectors.
- P3: Reduce warm-ping timeout from 30s to 15s (non-critical step)
- P3: Add trial status comment on forecast panel config

* fix(forecast): resolve ISO codes to country names, fix market detector, safe pre-push

P1 fixes from code review:
- CII ISO codes (IL, IR) now resolved to full country names (Israel, Iran)
  via country-codes.json. Prevents substring false positives (IL matching
  Chile) in event correlation. Uses word-boundary regex for matching.
- Market detector CII-to-theater mapping now uses entity graph traversal
  instead of broken theater-name substring matching. Iran correctly maps
  to Middle East theater via graph links.
- Pre-push hook no longer runs destructive git checkout on proto freshness
  failure. Reports mismatch and exits without modifying worktree.

* feat(forecast): add structured scenario pipeline and trace export

* fix(forecast): hydrate bootstrap and trim generated drift

* fix(forecast): keep required supply-chain contract updates

* fix(ci): add forecasts to cache-keys registry and regenerate proto

Add forecasts entry to BOOTSTRAP_CACHE_KEYS and BOOTSTRAP_TIERS in
cache-keys.ts to match api/bootstrap.js. Regenerate SupplyChain proto
to fix duplicate TransitDayCount and add riskSummary/riskReportAction.
 2026-03-15 15:57:22 +04:00
Elie Habib
33f15a26a9 refactor(chart): replace lightweight-charts with Canvas 2D transit chart (#1609) 
Remove TradingView's lightweight-charts dependency (~60KB) and replace
with a self-contained Canvas 2D implementation. Fixes:

- TradingView logo/branding and redirect links removed
- Chart data no longer hidden by legend labels (Cargo/Tanker overlapping March data)
- Shows last 60 days instead of full 6-month range
- Uses app theme CSS variables (--text-dim, --border-subtle, --accent-primary)
- Hover tooltips showing date + both values
- Bottom legend with colored dots and current values
- Highlighted endpoint dots for latest data
- Responds to theme-changed events
- ResizeObserver for responsive width
 2026-03-15 02:09:33 +04:00
Elie Habib
fe67111dc9 feat: harness engineering P0 - linting, testing, architecture docs (#1587) 
* feat: harness engineering P0 - linting, testing, architecture docs

Add foundational infrastructure for agent-first development:

- AGENTS.md: agent entry point with progressive disclosure to deeper docs
- ARCHITECTURE.md: 12-section system reference with source-file refs and ownership rule
- Biome 2.4.7 linter with project-tuned rules, CI workflow (lint-code.yml)
- Architectural boundary lint enforcing forward-only dependency direction (lint-boundaries.mjs)
- Unit test CI workflow (test.yml), all 1083 tests passing
- Fixed 9 pre-existing test failures (bootstrap sync, deploy-config headers, globe parity, redis mocks, geometry URL, import.meta.env null safety)
- Fixed 12 architectural boundary violations (types moved to proper layers)
- Added 3 missing cache tier entries in gateway.ts
- Synced cache-keys.ts with bootstrap.js
- Renamed docs/architecture.mdx to "Design Philosophy" with cross-references
- Deprecated legacy docs/Docs_To_Review/ARCHITECTURE.md
- Harness engineering roadmap tracking doc

* fix: address PR review feedback on harness-engineering-p0

- countries-geojson.test.mjs: skip gracefully when CDN unreachable
  instead of failing CI on network issues
- country-geometry-overrides.test.mts: relax timing assertion
  (250ms -> 2000ms) for constrained CI environments
- lint-boundaries.mjs: implement the documented api/ boundary check
  (was documented but missing, causing false green)

* fix(lint): scan api/ .ts files in boundary check

The api/ boundary check only scanned .js/.mjs files, missing the 25
sebuf RPC .ts edge functions. Now scans .ts files with correct rules:
- Legacy .js: fully self-contained (no server/ or src/ imports)
- RPC .ts: may import server/ and src/generated/ (bundled at deploy),
  but blocks imports from src/ application code

* fix(lint): detect import() type expressions in boundary lint

- Move AppContext back to app/app-context.ts (aggregate type that
  references components/services/utils belongs at the top, not types/)
- Move HappyContentCategory and TechHQ to types/ (simple enums/interfaces)
- Boundary lint now catches import('@/layer') expressions, not just
  from '@/layer' imports
- correlation-engine imports of AppContext marked boundary-ignore
  (type-only imports of top-level aggregate)
 2026-03-14 21:29:21 +04:00
Elie Habib
0383253a59 feat(supply-chain): chokepoint transit intelligence with 3 data sources (#1560) 
* feat(supply-chain): replace S&P Global with 3 free maritime data sources

Replace expensive S&P Global Maritime API with IMF PortWatch (vessel transit
counts), CorridorRisk (risk intelligence), and AISStream chokepoint crossing
counter. All external API calls run on Railway relay, Vercel reads Redis only.

- Add 4 new chokepoints (10 total): Cape of Good Hope, Gibraltar, Bosphorus, Dardanelles
- Add TransitSummary proto (field 14) with today counts, WoW%, 180d history, risk context
- Add D3 multi-line chart (tanker vs cargo) with expandable chokepoint cards
- Add crossing detection with enter+dwell+exit semantics, 30min cooldown, 5min min dwell
- Add PortWatch seed loop (6h), CorridorRisk seed loop (1h), transit seed loop (10min)
- Add canonical chokepoint ID map for cross-source name resolution
- 177 tests passing across 6 test files

* fix(supply-chain): address P2 review findings

- Discard partial PortWatch pagination results on mid-page failure (prevents
  truncated history with wrong WoW numbers cached for 6h)
- Rename "Transit today" to "24h" label (rolling 24h window, not calendar day)
- Fix chart label from "30d" to "180d" (matches actual PortWatch query range)
- Add 30s initial seed for chokepoint transits on relay cold start (prevents
  10min gap of zero transit data)

* feat(supply-chain): swap D3 chart for TradingView lightweight-charts

Replace hand-rolled D3 SVG transit chart with lightweight-charts v5 canvas
rendering for Bloomberg-quality time-series visualization.

- Add TransitChart helper class with mount/destroy lifecycle, theme listener,
  and autoSize support
- Use MutationObserver (not rAF) to mount chart after setContent debounce
- Clean up chart on tab switch, collapse, and re-render (no orphaned canvases)
- Respond to theme-changed events via chart.applyOptions()
- D3 stays for other 5 components (ProgressCharts, RenewableEnergy, etc.)

* feat(supply-chain): add geo coords and trade routes for 4 new chokepoints

Cherry-pick from PR #1511: Cape of Good Hope, Gibraltar, Bosphorus, and
Dardanelles map-layer coordinates and trade route definitions.

* fix(supply-chain): health.js v2->v4 key + double cache TTLs for missed seeds

- health.js chokepoints key was still v2, now v4 (matches handler + bootstrap)
- PortWatch TTL: 21600s (6h) -> 43200s (12h), seed interval stays 6h
- CorridorRisk TTL: 3600s (1h) -> 7200s (2h), seed interval stays 1h
- Ensures one missed seed run doesn't expire the key and cause empty data
 2026-03-14 14:20:49 +04:00
Elie Habib
3c69a3272f chore(deps): upgrade @vercel/analytics to v2.0.0 (#1432) 
Fixes "No route data found" warning in Vercel dashboard.
 2026-03-11 19:37:10 +04:00
Elie Habib
f26c1b3016 chore: bump version to 2.6.1 with changelog (#1410) 
Release 2.6.1 covering blog platform, country intelligence,
satellite imagery overhaul, and numerous fixes since 2.6.0.
 2026-03-11 10:51:10 +04:00
Elie Habib
26b55999e9 fix(blog): add blog build to build:full (Vercel production command) (#1403) 
Vercel uses build:full, not build. Without this, the blog Astro
site never gets built during production deploys, causing 404 at /blog.
 2026-03-11 08:33:48 +04:00
Elie Habib
b65464c0b2 feat(blog): add Astro blog at /blog with 16 SEO posts (#1401) 
* feat(blog): add Astro blog at /blog with 16 SEO-optimized posts

Adds a static Astro blog built during Vercel deploy and served at
worldmonitor.app/blog. Includes 16 marketing/SEO posts covering
features, use cases, and comparisons from customer perspectives.

- blog-site/: Astro static site with content collections, RSS, sitemap
- Vercel build pipeline: build:blog builds Astro and copies to public/blog/
- vercel.json: exclude /blog from SPA catch-all rewrite and no-cache headers
- vercel.json: ignoreCommand triggers deploy on blog-site/ changes
- Cache: /blog/_astro/* immutable, blog HTML uses Vercel defaults

* fix(blog): fix markdown lint errors in blog posts

Add blank lines around headings (MD022) and lists (MD032) across
all 16 blog post files to pass markdownlint checks.

* fix(ci): move ignoreCommand to script to stay under 256 char limit

Vercel schema validates ignoreCommand max length at 256 characters.
Move the logic to scripts/vercel-ignore.sh and reference it inline.

* fix(blog): address PR review findings

- Add blog sitemap to robots.txt for SEO discovery
- Use www.worldmonitor.app consistently (canonical domain)
- Clean public/blog/ before copy to prevent stale files
- Use npm ci for hermetic CI builds

* fix(blog): move blog dependency install to postinstall phase

Separates dependency installation from compilation. Blog deps are
now installed during npm install (postinstall hook), not during build.
 2026-03-11 08:20:56 +04:00
lspassos1
88282cc45f fix(deps): resolve all npm audit vulnerabilities (#1328) 
- Add serialize-javascript >=7.0.4 override to fix RCE via RegExp.flags
  (workbox-build -> @rollup/plugin-terser -> serialize-javascript chain)
- Bump markdownlint-cli2 ^0.20.0 -> ^0.21.0 to fix markdown-it ReDoS
- Run npm audit fix to resolve ajv, dompurify, minimatch, rollup, and
  fast-xml-parser advisories via lockfile updates

npm audit: 0 vulnerabilities
 2026-03-10 06:28:48 +04:00
Elie Habib
9129a3bbe3 chore: bump version to 2.6.0 (#1282) 
* chore: bump version to 2.6.0

* fix: non-null assertion for SearchModal list access
 2026-03-08 22:00:31 +04:00
Elie Habib
9772548d83 feat: add orbital surveillance layer with real-time satellite tracking (#1278) 
Track ~80-120 intelligence-relevant satellites on the 3D globe using CelesTrak
TLE data and client-side SGP4 propagation (satellite.js). Satellites render at
actual orbital altitude with country-coded colors, 15-min orbit trails, and
ground footprint projections.

Architecture: Railway seeds TLEs every 2h → Redis → Vercel CDN (1h cache) →
browser does SGP4 math every 3s (zero server cost for real-time movement).

- New relay seed loop (ais-relay.cjs) fetching military + resource groups
- New edge handler (api/satellites.js) with 10min cache + negative cache
- Frontend service with circuit breaker and propagation lifecycle
- GlobeMap integration: markers, trails (pathsData), footprints, tooltips
- Layer registry as globe-only "Orbital Surveillance" with i18n (21 locales)
- Full documentation at docs/ORBITAL_SURVEILLANCE.md with roadmap
- Fix pre-existing SearchModal TS error (non-null assertion)
 2026-03-08 21:55:46 +04:00
Elie Habib
0eec12706a feat(pro): localize landing page (21 languages) (#1187) 
* feat(pro): localize landing page with i18next (21 languages)

- Add i18next + browser language detection matching main app pattern
- Extract all hardcoded strings to locale keys (en.json)
- Lazy-load 20 non-English locale bundles via import.meta.glob
- Support RTL (Arabic) with automatic dir attribute
- Languages: en, ar, bg, cs, de, el, es, fr, it, ja, ko, nl, pl, pt, ro, ru, sv, th, tr, vi, zh

* fix(lint): exclude nested node_modules from markdownlint
 2026-03-07 13:52:07 +04:00
Elie Habib
8d83aa02eb fix(economic): guard against undefined BIS and spending data (#1162) 
* feat: premium panel gating, code cleanup, and backend simplifications

Recovered stranded changes from fix/desktop-premium-error-unification.

Premium gating:
- Add premium field ('locked'|'enhanced') to PanelConfig and LayerDefinition
- Panel.showLocked() with lock icon, CTA button, and _locked guard
- PRO badge for enhanced panels when no WM API key
- Exponential backoff auto-retry on showError() (15s→30s→60s→180s cap)
- Gate oref-sirens and telegram-intel panels behind WM API key
- Lock gpsJamming and iranAttacks layer toggles, badge ciiChoropleth
- Add tauri-titlebar drag region for custom titlebar

Code cleanup:
- Extract inline CSS from AirlineIntelPanel, WorldClockPanel to panels.css
- Remove unused showGeoError() from CountryBriefPage
- Remove dead geocodeFailed/retryBtn/closeBtn locale keys (20 files)
- Clean up var names and inline styles across 6 components

Backend:
- Remove seed-meta throttle from redis.ts (unnecessary complexity)
- Risk scores: call handler functions directly instead of raw Redis reads
- Update OpenRouter model to gpt-oss-safeguard-20b:nitro
- Add direct UCDP API fetching with version probing

Config:
- Remove titleBarStyle: Overlay from tauri.conf.json
- Add build:pro and build-sidecar-handlers to build:desktop
- Remove DXB/RUH from default aviation watchlist
- Simplify reverse-geocode (remove AbortController wrapper)

* fix: cast handler requests to any for API tsconfig compat

* fix: revert stale changes that conflict with merged PRs

Reverts files to main versions where old branch changes would
overwrite intentional fixes from PRs #1134, #1138, #1144, #1154:

- news/_shared.ts: keep gemini-2.5-flash model (not stale gpt-oss)
- redis.ts: keep seed-meta throttle from PR #1138
- reverse-geocode.ts: keep AbortController timeout from PR #1134
- CountryBriefPage.ts: keep showGeoError() from PR #1134
- country-intel.ts: keep showGeoError usage from PR #1134
- get-risk-scores.ts: revert non-existent imports
- watchlist.ts: keep DXB/RUH airports from PR #1144
- locales: restore geocodeFailed/retryBtn/closeBtn keys

* fix: neutralize language, parallel override loading, fetch timeout

- Rename conflict zone from "War" to "Border Conflict", intensity high→medium
- Rewrite description to factual language (no "open war" claim)
- Load country boundary overrides in parallel with main GeoJSON
- Neutralize comments/docs: reference Natural Earth source, remove political terms
- Add 60s timeout to Natural Earth fetch script (~24MB download)
- Add trailing newline to GeoJSON override file

* fix: restore caller messages in Panel errors and vessel expansion in popups

- Move UCDP direct-fetch cooldown after successful fetch to avoid
  suppressing all data for 10 minutes on a single failure
- Use caller-provided messages in showError/showRetrying instead of
  discarding them; respect autoRetrySeconds parameter
- Restore cluster-toggle click handler and expandable vessel list
  in military cluster popups
 2026-03-07 09:43:27 +04:00
JYR-AI
6745f47305 Variant/commodity (#1040) 
* commod variants

* mining map layers complete

* metal news feed

* commod variant final

* readme update

* fix: clean up commodity variant for merge readiness

- Remove duplicate FEEDS definition (central feeds.ts is source of truth)
- Remove duplicate inline ALLOWED_DOMAINS in rss-proxy.js (use shared module)
- Add 14 commodity RSS domains to shared/rss-allowed-domains.json
- Remove invalid geopoliticalBoundaries property (not in MapLayers type)
- Fix broken mobile-map-integration-harness imports
- Remove Substack credit link from app header
- Rename i18n key commod → commodity
- Extract mineralColor() helper for DRY color mapping
- Add XSS-safe tooltips for mining sites, processing plants, commodity ports
- Add missing interface fields (annualOutput, materials, capacityTpa, annualVolumeMt)
- Comment out unused COMMODITY_MINERS export
- Isolate commodity DeckGL changes from unrelated basemap refactor

* fix: hide commodity variant from selector until testing complete

Only show the commodity option in the variant switcher when the user
is already on the commodity variant (same pattern as happy variant).
Other variants (full, tech, finance) won't see the commodity link.

---------

Co-authored-by: jroachell <jianyin.roachell@siriusxm.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
 2026-03-06 09:41:35 +04:00
Elie Habib
fce836039b feat(map): migrate basemap from CARTO to self-hosted PMTiles on R2 (#1064) 
* feat(map): migrate basemap from CARTO to self-hosted PMTiles on Cloudflare R2

Replace CARTO tile provider (frequent 403 errors) with self-hosted PMTiles
served from Cloudflare R2. Uses @protomaps/basemaps for style generation
with OpenFreeMap as automatic fallback when VITE_PMTILES_URL is unset.

- Add pmtiles and @protomaps/basemaps dependencies
- Create src/config/basemap.ts for PMTiles protocol registration and style building
- Update DeckGLMap.ts to use PMTiles styles (non-happy variants)
- Fix fallback detection using data event instead of style.load
- Update SW cache rules: replace CARTO/MapTiler with PMTiles NetworkFirst
- Add Protomaps preconnect hints in index.html
- Bundle pmtiles + @protomaps/basemaps in maplibre chunk
- Upload 3.4GB world tiles (zoom 0-10) to R2 bucket worldmonitor-maps

* fix(map): use CDN custom domain maps.worldmonitor.app for PMTiles

Replace r2.dev URL with custom domain backed by Cloudflare CDN edge.
Update preconnect hint and .env.example with production URL.

* fix(map): harden PMTiles fallback detection to prevent false triggers

- Require 2+ network errors before triggering OpenFreeMap fallback
- Use persistent data listener instead of once (clears timeout on first tile load)
- Increase fallback timeout to 10s for PMTiles header + initial tile fetch
- Add console.warn for map errors to aid debugging
- Remove redundant style.load listener (fires immediately for inline styles)

* feat(settings): add Map Tile Provider selector in settings

Add dropdown in Settings → Map section to switch between:
- Auto (PMTiles → OpenFreeMap fallback)
- PMTiles (self-hosted)
- OpenFreeMap
- CARTO

Choice persists in localStorage and reloads basemap instantly.

* fix(map): make OSS-friendly — default to free OpenFreeMap, hide PMTiles when unconfigured

- Default to OpenFreeMap when VITE_PMTILES_URL is unset (zero config for OSS users)
- Hide PMTiles/Auto options from settings dropdown when no PMTiles URL configured
- If user previously selected PMTiles but env var is removed, gracefully fall back
- Remove production URL from .env.example to avoid exposing hosted tiles
- Add docs link for self-hosting PMTiles in .env.example

* docs: add map tile provider documentation to README and MAP_ENGINE.md

Document the tile provider system (OpenFreeMap, CARTO, PMTiles) in
MAP_ENGINE.md with self-hosting instructions, fallback behavior, and
OSS-friendly defaults. Update README to reference tile providers in
the feature list, tech stack, and environment variables table.

* fix: resolve rebase conflicts and fix markdown lint errors

- Restore OSS-friendly basemap defaults (MAP_PROVIDER_OPTIONS as IIFE,
  getMapProvider with hasTilesUrl check)
- Fix markdown lint: add blank lines after ### headings in README
- Reconcile UnifiedSettings import with MAP_PROVIDER_OPTIONS constant
 2026-03-06 08:40:14 +04:00
Elie Habib
29ef8eae2f docs: update README with accurate counts and 9 new feature sections (#1071) 
- Fix stale counts: 170+ feeds → 435+, 15 bootstrap keys → 38,
  28+ data sources → 31, 20+ search types → 24, panel counts
- Add Aviation Intelligence Panel documentation
- Add Customizable Market Watchlist section
- Add News Importance Scoring algorithm details
- Add Railway Seed Data Pipeline table (21 cron jobs)
- Add SmartPollLoop adaptive polling documentation
- Expand Prediction Markets with 4-tier fetch strategy
- Add Iran conflict monitoring layer details
- Add Mobile search sheet and FAB section
- Expand Regression Testing section (30 files, 554 tests)
- Expand Bootstrap Hydration with full 38-key tier listing
- Bump version 2.5.24 → 2.5.25
 2026-03-05 23:40:37 +04:00
Mert Efe Şensoy
f771114522 feat: aviation monitoring layer with flight tracking, airline intel panel, and news feeds (#907) 
* feat: Implement comprehensive aviation monitoring service with flight search, status, news, and tracking.

* feat: Introduce Airline Intelligence Panel with aviation data tabs, map components, and localization.

* feat: Implement DeckGL-based map for advanced visualization, D3/SVG fallback, i18n support, and aircraft tracking.

* Update server/worldmonitor/aviation/v1/get-carrier-ops.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update server/worldmonitor/aviation/v1/search-flight-prices.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update server/worldmonitor/aviation/v1/track-aircraft.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update server/worldmonitor/aviation/v1/get-airport-ops-summary.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update proto/worldmonitor/aviation/v1/position_sample.proto

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update server/worldmonitor/aviation/v1/list-airport-flights.ts

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update proto/worldmonitor/aviation/v1/price_quote.proto

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: Add server-side endpoints for aviation news and aircraft tracking, and introduce a new DeckGLMap component for map visualization.

* Update server/worldmonitor/aviation/v1/list-airport-flights.ts

The cache key for listAirportFlights excludes limit, but the upstream fetch/simulated generator uses limit to determine how many flights to return. If the first request within TTL uses a small limit, larger subsequent requests will be incorrectly capped until cache expiry. Include limit (or a normalized bucket/max) in cacheKey, or always fetch/cache a fixed max then slice per request.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* Update server/worldmonitor/aviation/v1/get-flight-status.ts

getFlightStatus accepts origin, but cacheKey does not include it. This can serve cached results from an origin-less query to an origin-filtered query (or vice versa). Add origin (normalized) to the cache key or apply filtering after fetch to ensure cache correctness.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: Implement DeckGL map for advanced visualization and new aviation data services.

* fix(aviation): prevent cache poisoning and keyboard shortcut in inputs

- get-carrier-ops: move minFlights filter post-cache to avoid cache
  fragmentation (different callers sharing cached full result)
- AviationCommandBar: guard Ctrl+J shortcut so it does not fire when
  focus is inside an INPUT or TEXTAREA element

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* feat: introduce AviationCommandBar component for parsing user commands, fetching aviation data, and displaying results.

* feat: Implement aircraft tracking service with OpenSky and simulated data sources.

* feat: introduce DeckGLMap component for WebGL-accelerated map visualizations using deck.gl and maplibre-gl.

* fix(aviation): address code review findings for PR #907

Proto: add missing (sebuf.http.query) annotations on all GET request
fields across 6 proto files; add currency/market fields to
SearchFlightPricesRequest.

Server: add parseStringArray to aviation _shared.ts and apply to
get-airport-ops-summary, get-carrier-ops, list-aviation-news handlers
to prevent crash on comma-separated query params; remove leaked API
token from URL params in travelpayouts_data; fix identical simulated
flight statuses in list-airport-flights; remove unused endDate var;
normalize cache key entity casing in list-aviation-news.

Client: refactor AirlineIntelPanel to extend Panel base class and
register in DEFAULT_PANELS for full/tech/finance variants; fix
AviationCommandBar reference leak with proper destroy() cleanup in
panel-layout; rename priceUsd→priceAmount in display type and all
usages; change auto-refresh to call refresh() instead of loadOps().

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: introduce aviation command bar component with aircraft tracking and flight information services.

* feat: Add `AirlineIntelPanel` component for displaying airline operations, flights, carriers, tracking, news, and prices in a tabbed interface.

* feat: Add endpoints for listing airport flights and fetching aviation news.

* Update proto/worldmonitor/aviation/v1/search_flight_prices.proto

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

* feat: Add server endpoint for listing airport flights and client-side MapPopup types and utilities.

* feat: Introduce MapPopup component with support for various data types and responsive positioning for map features.

* feat: Add initial English localization file (en.json).

* fix(aviation): address PR review findings across aviation stack

- Add User-Agent header to Travelpayouts provider (server convention)
- Use URLSearchParams for API keys instead of raw URL interpolation
- Add input length validation on flightNumber (max 10 chars)
- Replace regex XML parsing with fast-xml-parser in aviation news
- Fix (f as any)._airport type escape with typed Map<FI, string>
- Extract DEFAULT_WATCHED_AIRPORTS constant from hardcoded arrays
- Use event delegation for AirlineIntelPanel price search listener
- Add bootstrap hydration key for flight delays
- Bump OpenSky cache TTL to 120s (anonymous tier rate limit)
- Match DeckGLMap aircraft poll interval to server cache (120s)
- Fix GeoJSON polygon winding order (shoelace check + auto-reversal)

* docs: add aviation env vars to .env.example

AVIATIONSTACK_API, ICAO_API_KEY, TRAVELPAYOUTS_API_TOKEN

* feat: Add aviation news listing API and introduce shared RSS allowed domains.

* fix: add trailing newline to rss-allowed-domains.json, remove unused ringIsClockwise

---------

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>
Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
 2026-03-04 21:09:37 +04:00
Elie Habib
034ab9916f feat(globe): add interactive 3D globe view with 28 live data layers (#926) 
* feat(globe): add 3D globe view powered by globe.gl

Replicate the Sentinel.axonia.us globe locally and expose it via Settings.

- Add GlobeMap.ts: new globe.gl v2 component with night-sky starfield,
  earth topobathy texture, specular water map, atmosphere glow, auto-rotate
  (pauses on interaction, resumes after 60 s), and HTML marker layer for
  conflict zones, intel hotspots, and other data categories
- Update MapContainer with switchToGlobe() / switchToFlat() runtime
  methods and isGlobeMode() query; constructor accepts preferGlobe param
- Wire globe toggle in UnifiedSettings General tab (MAP section);
  persisted to worldmonitor-map-mode via loadFromStorage/saveToStorage
- Add mapMode storage key to STORAGE_KEYS
- Download earth textures to public/textures/ (topo-bathy, night-sky,
  water specular, day)
- Add globe.gl ^2.45.0 and @types/three dependencies
- Add globe CSS + @keyframes globe-pulse for pulsing conflict markers

* feat(globe): wire region selector & CMD+K navigation to 3D globe

* feat(globe): add zoom controls, layer panel, marker tooltips; fix Vercel build

* feat(globe): expand to all 28 world-variant layers with live data rendering

* refactor(globe): use proper keyof MapLayers types

* fix(globe): route AIS/flight data to globe, implement ship traffic markers, hide dayNight toggle

- MapContainer: add globe guard to setAisData and setFlightDelays (data was silently dropped)
- GlobeMap: implement setAisData with AisDisruptionMarker (typed, no any casts); renders
  disruption events with severity-colored ship icons and full tooltip (name/type/severity)
- GlobeMap: three-point dayNight suppression — disabled in initGlobe(), overridden in
  setLayers(), ignored in enableLayer(); toggle removed from layer panel UI
- MapContainer: add globe guards to 5 happy-variant setters (P3: keep no-op stubs in globe)
- Add tests/globe-2d-3d-parity.test.mjs: 13 static-analysis tests covering routing,
  AIS marker fields, and dayNight suppression (all passing)
 2026-03-03 21:08:06 +04:00
Nicolas Dos Santos
7373135978 fix: resolve build errors and fullscreen overlay bugs (#857, #859, #860, #829) 
Add missing supercluster, preact, @types/geojson, and @types/supercluster dependencies. Remove DOM reparenting in LiveNewsPanel fullscreen toggle that caused iframe reloads and channel switching. Hide sibling panels and map overlays when a panel is in fullscreen mode.
 2026-03-03 09:56:25 +04:00
Elie Habib
d1318781ff docs(readme): update stats, add 8 new sections, bump to v2.5.24 (#874) 
Correct all stale numbers to match current codebase:
- Languages: 16 → 19 (added Czech, Greek, Korean)
- RSS feeds: 150+ → 170+, live channel pool: 30+ → 70+
- Airports: 128 → 107, AviationStack: 114 → 40
- Hotspots: 74 → 217, proto domains: 20 → 22
- Telegram: 27 → 26, OREF locations: 1,478 → 1,480
- Panel counts: 45/31/31/8 → 47/35/33/10

Add 8 new documentation sections:
- Bootstrap Hydration (2-tier parallel pre-fetch)
- Breaking News Alert Pipeline (5 origins)
- Cross-Stream Correlation Engine (14 signal types)
- Adaptive Refresh Scheduling (backoff, jitter, throttle)
- Localization Architecture (bundles, boost, RTL, fonts)
- Intelligence Analysis Tradecraft (SATs, ACH, gap awareness)
- Client-Side Circuit Breakers (IndexedDB persistence)
- Programmatic API Access (api.worldmonitor.app)

Expand Happy Monitor with humanity counters, conservation,
renewables, and giving detail. Add negative caching docs.
Bump version 2.5.23 → 2.5.24.
 2026-03-03 08:36:49 +04:00
Elie Habib
e14af08f2d fix(desktop): resolve sidecar 401s, variant lock, and registration form (#v2.5.23) (#709) 
- Sidecar 401 fix: inject trusted localhost Origin on requests passed to
  handler modules. The handler's validateApiKey() was seeing empty Origin
  (stripped by toHeaders) + no API key → 401 for ALL desktop API calls.
- Variant fix: check localStorage FIRST when running in Tauri desktop,
  so .env.local VITE_VARIANT doesn't override user's variant selection.
- Registration: force-show form for email delivery testing.
- Bump version to 2.5.23.
 2026-03-02 02:08:57 +04:00
Elie Habib
6adfda8061 chore: bump version to 2.5.22 & comprehensive README update (#706) 
Bump version 2.5.21 → 2.5.22 across package.json, Cargo.toml, and
tauri.conf.json.

README: document 15+ recently shipped features that were missing from
the README — AI Deduction panel, Headline Memory (RAG), server-side
feed aggregation, Gulf Economies panel, TV Mode, mobile map with touch
gestures, fullscreen live video, 18+ HLS channels, breaking news
click-through, badge animation toggle, cache purge admin endpoint,
locale-aware feed boost, OREF Redis persistence + 1,478 Hebrew→English
translations, and Oceania region tab. Update PostHog → Vercel Analytics.
Add 21 new completed roadmap items.
 2026-03-02 01:24:20 +04:00
Elie Habib
078a239ceb feat(live-news): add CNN & CNBC HLS streams via sidecar proxy (#682) 
* feat(live-news): add CNN & CNBC HLS streams via sidecar proxy (desktop only)

Add /api/hls-proxy route to sidecar that proxies HLS manifests and
segments from allowlisted CDN hosts, injecting the required Referer
header that browsers cannot set. Rewrites m3u8 URLs so all segments
and encryption keys also route through the proxy.

Desktop gets native <video> HLS playback for CNN and CNBC; web falls
through to YouTube as before (no bandwidth cost on Vercel).

* fix(types): add missing @types/dompurify dev dependency
 2026-03-01 21:06:18 +04:00
Chris Chen
a7efa7dda8 feat: implement deduct situation feature (#636) (#642) 
* Add Security Advisories panel with government travel alerts (#460)

* feat: add Security Advisories panel with government travel advisory feeds

Adds a new panel aggregating travel/security advisories from official
government foreign affairs agencies (US State Dept, AU DFAT Smartraveller,
UK FCDO, NZ MFAT). Advisories are categorized by severity level
(Do Not Travel, Reconsider, Caution, Normal) with filter tabs by
source country. Includes summary counts, auto-refresh, and persistent
caching via the existing data-freshness system.


* chore: update package-lock.json


* fix: event delegation, localization, and cleanup for SecurityAdvisories panel

P1 fixes:
- Use event delegation on this.content (bound once in constructor) instead
  of direct addEventListener after each innerHTML replacement — prevents
  memory leaks and stale listener issues on re-render
- Use setContent() consistently instead of mixing with this.content.innerHTML
- Add securityAdvisories translations to all 16 non-English locale files
  (panels name, component strings, common.all key)
- Revert unrelated package-lock.json version bump

P2 fixes:
- Deduplicate loadSecurityAdvisories — loadIntelligenceData now calls the
  shared method instead of inlining duplicate fetch+set logic
- Add Accept header to fetch calls for better content negotiation

* feat(advisories): add US embassy alerts, CDC, ECDC, and WHO health feeds

Adds 21 new advisory RSS feeds:
- 13 US Embassy per-country security alerts (TH, AE, DE, UA, MX, IN, PK, CO, PL, BD, IT, DO, MM)
- CDC Travel Notices
- 5 ECDC feeds (epidemiological, threats, risk assessments, avian flu, publications)
- 2 WHO feeds (global news, Africa emergencies)

Panel gains a Health filter tab for CDC/ECDC/WHO sources.
All new domains added to RSS proxy allowlist.
i18n "health" key added across all 17 locales.

* feat(cache): add negative-result caching to cachedFetchJson (#466)

When upstream APIs return errors (HTTP 403, 429, timeout), fetchers
return null. Previously null results were not cached, causing repeated
request storms against broken APIs every refresh cycle.

Now caches a sentinel value ('__WM_NEG__') with a short 2-minute TTL
on null results. Subsequent requests within that window get null
immediately without hitting upstream. Thrown errors (transient) skip
sentinel caching and retry immediately.

Also filters sentinels from getCachedJsonBatch pipeline reads and fixes
theater posture coalescing test (expected 2 OpenSky fetches for 2
theater query regions, not 1).

* feat: convert 52 API endpoints from POST to GET for edge caching (#468)

* feat: convert 52 API endpoints from POST to GET for edge caching

Convert all cacheable sebuf RPC endpoints to HTTP GET with query/path
parameters, enabling CDN edge caching to reduce costs. Flatten nested
request types (TimeRange, PaginationRequest, BoundingBox) into scalar
query params. Add path params for resource lookups (GetFredSeries,
GetHumanitarianSummary, GetCountryStockIndex, GetCountryIntelBrief,
GetAircraftDetails). Rewrite router with hybrid static/dynamic matching
for path param support.

Kept as POST: SummarizeArticle, ClassifyEvent, RecordBaselineSnapshot,
GetAircraftDetailsBatch, RegisterInterest.

Generated with sebuf v0.9.0 (protoc-gen-ts-client, protoc-gen-ts-server).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add rate_limited field to market response protos

The rateLimited field was hand-patched into generated files on main but
never declared in the proto definitions. Regenerating wiped it out,
breaking the build. Now properly defined in both ListEtfFlowsResponse
and ListMarketQuotesResponse protos.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove accidentally committed .planning files

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* feat: add Cloudflare edge caching infrastructure for api.worldmonitor.app (#471)

Route web production RPC traffic through api.worldmonitor.app via fetch
interceptor (installWebApiRedirect). Add default Cache-Control headers
(s-maxage=300, stale-while-revalidate=60) on GET 200 responses, with
no-store override for real-time endpoints (vessel snapshot). Update CORS
to allow GET method. Skip Vercel bot middleware for API subdomain using
hostname check (non-spoofable, replacing CF-Ray header approach). Update
desktop cloud fallback to route through api.worldmonitor.app.

* fix(beta): eagerly load T5-small model when beta mode is enabled

BETA_MODE now couples the badge AND model loading — the summarization-beta
model starts loading on startup instead of waiting for the first summarization call.

* fix: move 5 path-param endpoints to query params for Vercel routing (#472)

Vercel's `api/[domain]/v1/[rpc].ts` captures one dynamic segment.
Path params like `/get-humanitarian-summary/SA` add an extra segment
that has no matching route file, causing 404 on both OPTIONS preflight
and direct requests. These endpoints were broken in production.

Changes:
- Remove `{param}` from 5 service.proto HTTP paths
- Add `(sebuf.http.query)` annotations to request message fields
- Update generated client/server code to use URLSearchParams
- Update OpenAPI specs (YAML + JSON) to declare query params
- Add early-return guards in 4 handlers for missing required params
- Add happy.worldmonitor.app to runtime.ts redirect hosts

Affected endpoints:
- GET /api/conflict/v1/get-humanitarian-summary?country_code=SA
- GET /api/economic/v1/get-fred-series?series_id=T10Y2Y&limit=120
- GET /api/market/v1/get-country-stock-index?country_code=US
- GET /api/intelligence/v1/get-country-intel-brief?country_code=US
- GET /api/military/v1/get-aircraft-details?icao24=a12345

* fix(security-advisories): route feeds through RSS proxy to avoid CORS blocks (#473)

- Advisory feeds were fetched directly from the browser, hitting CORS
  on all 21 feeds (US State Dept, AU Smartraveller, US Embassies, ECDC,
  CDC, WHO). Route through /api/rss-proxy on web, keep proxyUrl for desktop.
- Fix double slash in ECDC Avian Influenza URL (323//feed → 323/feed)
- Add feeds.news24.com to RSS proxy allowlist (was returning 403)

* feat(cache): tiered edge Cache-Control aligned to upstream TTLs (#474)

* fix: move 5 path-param endpoints to query params for Vercel routing

Vercel's `api/[domain]/v1/[rpc].ts` captures one dynamic segment.
Path params like `/get-humanitarian-summary/SA` add an extra segment
that has no matching route file, causing 404 on both OPTIONS preflight
and direct requests. These endpoints were broken in production.

Changes:
- Remove `{param}` from 5 service.proto HTTP paths
- Add `(sebuf.http.query)` annotations to request message fields
- Update generated client/server code to use URLSearchParams
- Update OpenAPI specs (YAML + JSON) to declare query params
- Add early-return guards in 4 handlers for missing required params
- Add happy.worldmonitor.app to runtime.ts redirect hosts

Affected endpoints:
- GET /api/conflict/v1/get-humanitarian-summary?country_code=SA
- GET /api/economic/v1/get-fred-series?series_id=T10Y2Y&limit=120
- GET /api/market/v1/get-country-stock-index?country_code=US
- GET /api/intelligence/v1/get-country-intel-brief?country_code=US
- GET /api/military/v1/get-aircraft-details?icao24=a12345

* feat(cache): add tiered edge Cache-Control aligned to upstream TTLs

Replace flat s-maxage=300 with 5 tiers (fast/medium/slow/static/no-store)
mapped per-endpoint to respect upstream Redis TTLs. Adds stale-if-error
resilience headers and X-No-Cache plumbing for future degraded responses.
X-Cache-Tier debug header gated behind ?_debug query param.

* fix(tech): use rss() for CISA feed, drop build from pre-push hook (#475)

- CISA Advisories used dead rss.worldmonitor.app domain (404), switch to rss() helper
- Remove Vite build from pre-push hook (tsc already catches errors)

* fix(desktop): enable click-to-play YouTube embeds + CISA feed fixes (#476)

* fix(tech): use rss() for CISA feed, drop build from pre-push hook

- CISA Advisories used dead rss.worldmonitor.app domain (404), switch to rss() helper
- Remove Vite build from pre-push hook (tsc already catches errors)

* fix(desktop): enable click-to-play for YouTube embeds in WKWebView

WKWebView blocks programmatic autoplay in cross-origin iframes regardless
of allow attributes, Permissions-Policy, mute-first retries, or secure
context. Documented all 10 approaches tested in docs/internal/.

Changes:
- Switch sidecar embed origin from 127.0.0.1 to localhost (secure context)
- Add MutationObserver + retry chain as best-effort autoplay attempts
- Use postMessage('*') to fix tauri://localhost cross-origin messaging
- Make sidecar play overlay non-interactive (pointer-events:none)
- Fix .webcam-iframe pointer-events:none blocking clicks in grid view
- Add expand button to grid cells for switching to single view on desktop
- Add http://localhost:* to CSP frame-src in index.html and tauri.conf.json

* fix(gateway): convert stale POST requests to GET for backwards compat (#477)

Stale cached client bundles still send POST to endpoints converted to
GET in PR #468, causing 404s. The gateway now parses the POST JSON body
into query params and retries the match as GET.

* feat(proxy): add Cloudflare edge caching for proxy.worldmonitor.app (#478)

Add CDN-Cache-Control headers to all proxy endpoints so Cloudflare can
cache responses at the edge independently of browser Cache-Control:

- RSS: 600s edge + stale-while-revalidate=300 (browser: 300s)
- UCDP: 3600s edge (matches browser)
- OpenSky: 15s edge (browser: 30s) for fresher flight data
- WorldBank: 1800s/86400s edge (matches browser)
- Polymarket: 120s edge (matches browser)
- Telegram: 10s edge (matches browser)
- AIS snapshot: 2s edge (matches browser)

Also fixes:
- Vary header merging: sendCompressed/sendPreGzipped now merge existing
  Vary: Origin instead of overwriting, preventing cross-origin cache
  poisoning at the edge
- Stale fallback responses (OpenSky, WorldBank, Polymarket, RSS) now
  set Cache-Control: no-store + CDN-Cache-Control: no-store to prevent
  edge caching of degraded responses
- All no-cache branches get CDN-Cache-Control: no-store
- /opensky-reset gets no-store (state-changing endpoint)

* fix(sentry): add noise filters for 4 unresolved issues (#479)

- Tighten AbortError filter to match "AbortError: The operation was aborted"
- Filter "The user aborted a request" (normal navigation cancellation)
- Filter UltraViewer service worker injection errors (/uv/service/)
- Filter Huawei WebView __isInQueue__ injection

* feat: configurable VITE_WS_API_URL + harden POST→GET shim (#480)

* fix(gateway): harden POST→GET shim with scalar guard and size limit

- Only convert string/number/boolean values to query params (skip objects,
  nested arrays, __proto__ etc.) to prevent prototype pollution vectors
- Skip body parsing for Content-Length > 1MB to avoid memory pressure

* feat: make API base URL configurable via VITE_WS_API_URL

Replace hardcoded api.worldmonitor.app with VITE_WS_API_URL env var.
When empty, installWebApiRedirect() is skipped entirely — relative
/api/* calls stay on the same domain (local installs). When set,
browser fetch is redirected to that URL.

Also adds VITE_WS_API_URL and VITE_WS_RELAY_URL hostnames to
APP_HOSTS allowlist dynamically.

* fix(analytics): use greedy regex in PostHog ingest rewrites (#481)

Vercel's :path* wildcard doesn't match trailing slashes that
PostHog SDK appends (e.g. /ingest/s/?compression=...), causing 404s.
Switch to :path(.*) which matches all path segments including
trailing slashes. Ref: PostHog/posthog#17596

* perf(proxy): increase AIS snapshot edge TTL from 2s to 10s (#482)

With 20k requests/30min (60% of proxy traffic) and per-PoP caching,
a 2s edge TTL expires before the next request from the same PoP arrives,
resulting in near-zero cache hits. 10s allows same-PoP dedup while
keeping browser TTL at 2s for fresh vessel positions.

* fix(markets): commodities panel showing stocks instead of commodities (#483)

The shared circuit breaker (cacheTtlMs: 0) cached the stocks response,
then the stale-while-revalidate path returned that cached stocks data
for the subsequent commodities fetch. Skip SWR when caching is disabled.

* feat(gateway): complete edge cache tier coverage + degraded-response policy (#484)

- Add 11 missing GET routes to RPC_CACHE_TIER map (8 slow, 3 medium)
- Add response-headers side-channel (WeakMap) so handlers can signal
  X-No-Cache without codegen changes; wire into military-flights and
  positive-geo-events handlers on upstream failure
- Add env-controlled per-endpoint tier override (CACHE_TIER_OVERRIDE_*)
  for incident response rollback
- Add VITE_WS_API_URL hostname allowlist (*.worldmonitor.app + localhost)
- Fix fetch.bind(globalThis) in positive-events-geo.ts (deferred lambda)
- Add CI test asserting every generated GET route has an explicit cache
  tier entry (prevents silent default-tier drift)

* chore: bump version to 2.5.20 + changelog

Covers PRs #452–#484: Cloudflare edge caching, commodities SWR fix,
security advisories panel, settings redesign, 52 POST→GET migrations.

* fix(rss): remove stale indianewsnetwork.com from proxy allowlist (#486)

Feed has no <pubDate> fields and latest content is from April 2022.
Not referenced in any feed config — only in the proxy domain allowlist.

* feat(i18n): add Korean (한국어) localization (#487)

- Add ko.json with all 1606 translation keys matching en.json structure
- Register 'ko' in SUPPORTED_LANGUAGES, LANGUAGES display array, and locale map
- Korean appears as 🇰🇷 한국어 in the language dropdown

* feat: add Polish tv livestreams (#488)

* feat(rss): add Axios (api.axios.com/feed) as US news source (#494)

Add api.axios.com to proxy allowlist and CSP connect-src, register
Axios feed under US category as Tier 2 mainstream source.

* perf: bootstrap endpoint + polling optimization (#495)

* perf: bootstrap endpoint + polling optimization (phases 3-4)

Replace 15+ individual RPC calls on startup with a single /api/bootstrap
batch call that fetches pre-cached data from Redis. Consolidate 6 panel
setInterval timers into the central RefreshScheduler for hidden-tab
awareness (10x multiplier) and adaptive backoff (up to 4x for unchanged
data). Convert IntelligenceGapBadge from 10s polling to event-driven
updates with 60s safety fallback.

* fix(bootstrap): inline Redis + cache keys in edge function

Vercel Edge Functions cannot resolve cross-directory TypeScript imports
from server/_shared/. Inline getCachedJsonBatch and BOOTSTRAP_CACHE_KEYS
directly in api/bootstrap.js. Add sync test to ensure inlined keys stay
in sync with the canonical server/_shared/cache-keys.ts registry.

* test: add Edge Function module isolation guard for all api/*.js files

Prevents any Edge Function from importing from ../server/ or ../src/
which breaks Vercel builds. Scans all 12 non-helper Edge Functions.

* fix(bootstrap): read unprefixed cache keys on all environments

Preview deploys set VERCEL_ENV=preview which caused getKeyPrefix() to
prefix Redis keys with preview:<sha>:, but handlers only write to
unprefixed keys on production. Bootstrap is a read-only consumer of
production cache — always read unprefixed keys.

* fix(bootstrap): wire sectors hydration + add coverage guard

- Wire getHydratedData('sectors') in data-loader to skip Yahoo Finance
  fetch when bootstrap provides sector data
- Add test ensuring every bootstrap key has a getHydratedData consumer
  — prevents adding keys without wiring them

* fix(server): resolve 25 TypeScript errors + add server typecheck to CI

- _shared.ts: remove unused `delay` variable
- list-etf-flows.ts: add missing `rateLimited` field to 3 return literals
- list-market-quotes.ts: add missing `rateLimited` field to 4 return literals
- get-cable-health.ts: add non-null assertions for regex groups and array access
- list-positive-geo-events.ts: add non-null assertion for array index
- get-chokepoint-status.ts: add required fields to request objects
- CI: run `typecheck:api` (tsconfig.api.json) alongside `typecheck` to catch
  server/ TS errors before merge

* feat(military): server-side military bases 125K + rate limiting (#496)

* feat(military): server-side military bases with 125K entries + rate limiting (#485)

Migrate military bases from 224 static client-side entries to 125,380
server-side entries stored in Redis GEO sorted sets, served via
bbox-filtered GEOSEARCH endpoint with server-side clustering.

Data pipeline:
- Pizzint/Polyglobe: 79,156 entries (Supabase extraction)
- OpenStreetMap: 45,185 entries
- MIRTA: 821 entries
- Curated strategic: 218 entries
- 277 proximity duplicates removed

Server:
- ListMilitaryBases RPC with GEOSEARCH + HMGET + tier/filter/clustering
- Antimeridian handling (split bbox queries)
- Blue-green Redis deployment with atomic version pointer switch
- geoSearchByBox() + getHashFieldsBatch() helpers in redis.ts

Security:
- @upstash/ratelimit: 60 req/min sliding window per IP
- IP spoofing fix: prioritize x-real-ip (Vercel-injected) over x-forwarded-for
- Require API key for non-browser requests (blocks unauthenticated curl/scripts)
- Input validation: allowlisted types/kinds, regex country, clamped bbox/zoom

Frontend:
- Viewport-driven loading with bbox quantization + debounce
- Server-side grid clustering at low zoom levels
- Enriched popup with kind, category badges (airforce/naval/nuclear/space)
- Static 224 bases kept as search fallback + initial render

* fix(military): fallback to production Redis keys in preview deployments

Preview deployments prefix Redis keys with `preview:{sha}:` but military
bases data is seeded to unprefixed (production) keys. When the prefixed
`military:bases:active` key is missing, fall back to the unprefixed key
and use raw (unprefixed) keys for geo/meta lookups.

* fix: remove unused 'remaining' destructure in rate-limit (TS6133)

* ci: add typecheck:api to pre-push hook to catch server-side TS errors

* debug(military): add X-Bases-Debug response header for preview diagnostics

* fix(bases): trigger initial server fetch on map load

fetchServerBases() was only called on moveend — if the user
never panned/zoomed, the API was never called and only the 224
static fallback bases showed.

* perf(military): debounce base fetches + upgrade edge cache to static tier (#497)

- Add 300ms debounce on moveend to prevent rapid pan flooding
- Fixes stale-bbox bug where pendingFetch returns old viewport data
- Upgrade edge cache tier from medium (5min) to static (1hr) — bases are
  static infrastructure, aligned with server-side cachedFetchJson TTL
- Keep error logging in catch blocks for production diagnostics

* fix(cyber): make GeoIP centroid fallback jitter deterministic (#498)

Replace Math.random() jitter with DJB2 hash seeded by the threat
indicator (IP/URL), so the same threat always maps to the same
coordinates across requests while different threats from the same
country still spread out.

Closes #203

Co-authored-by: Chris Chen <fuleinist@users.noreply.github.com>

* fix: use cross-env for Windows-compatible npm scripts (#499)

Replace direct `VAR=value command` syntax with cross-env/cross-env-shell
so dev, build, test, and desktop scripts work on Windows PowerShell/CMD.

Co-authored-by: facusturla <facusturla@users.noreply.github.com>

* feat(live-news): add CBC News to optional North America channels (#502)

YouTube handle @CBCNews with fallback video ID 5vfaDsMhCF4.

* fix(bootstrap): harden hydration cache + polling review fixes (#504)

- Filter null/undefined values before storing in hydration cache to
  prevent future consumers using !== undefined from misinterpreting
  null as valid data
- Debounce wm:intelligence-updated event handler via requestAnimationFrame
  to coalesce rapid alert generation into a single render pass
- Include alert IDs in StrategicRiskPanel change fingerprint so content
  changes are detected even when alert count stays the same
- Replace JSON.stringify change detection in ServiceStatusPanel with
  lightweight name:status fingerprint
- Document max effective refresh interval (40x base) in scheduler

* fix(geo): tokenization-based keyword matching to prevent false positives (#503)

* fix(geo): tokenization-based keyword matching to prevent false positives

Replace String.includes() with tokenization-based Set.has() matching
across the geo-tagging pipeline. Prevents false positives like "assad"
matching inside "ambassador" and "hts" matching inside "rights".

- Add src/utils/keyword-match.ts as single source of truth
- Decompose possessives/hyphens ("Assad's" → includes "assad")
- Support multi-word phrase matching ("white house" as contiguous)
- Remove false-positive-prone DC keywords ('house', 'us ')
- Update 9 consumer files across geo-hub, map, CII, and asset systems
- Add 44 tests covering false positives, true positives, edge cases

Co-authored-by: karim <mirakijka@gmail.com>
Fixes #324

* fix(geo): add inflection suffix matching + fix test imports

Address code review feedback:

P1a: Add suffix-aware matching for plurals and demonyms so existing
keyword lists don't regress (houthi→houthis, ukraine→ukrainian,
iran→iranian, israel→israeli, russia→russian, taiwan→taiwanese).
Uses curated suffix list + e-dropping rule to avoid false positives.

P1b: Expand conflictTopics arrays in DeckGLMap and Map with demonym
forms so "Iranian senate..." correctly registers as conflict topic.

P2: Replace inline test functions with real module import via tsx.
Tests now exercise the production keyword-match.ts directly.

* fix: wire geo-keyword tests into test:data command

The .mts test file wasn't covered by `node --test tests/*.test.mjs`.
Add `npx tsx --test tests/*.test.mts` so test:data runs both suites.

* fix: cross-platform test:data + pin tsx in devDependencies

- Use tsx as test runner for both .mjs and .mts (single invocation)
- Removes ; separator which breaks on Windows cmd.exe
- Add tsx to devDependencies so it works in offline/CI environments

* fix(geo): multi-word demonym matching + short-keyword suffix guard

- Add wordMatches() for suffix-aware phrase matching so "South Korean"
  matches keyword "south korea" and "North Korean" matches "north korea"
- Add MIN_SUFFIX_KEYWORD_LEN=4 guard so short keywords like "ai", "us",
  "hts" only do exact-match (prevents "ais"→"ai", "uses"→"us" false positives)
- Add 5 new tests covering both fixes (58 total, all passing)

* fix(geo): support plural demonyms in keyword matching

Add compound suffixes (ians, eans, ans, ns, is) to handle plural
demonym forms like "Iranians"→"iran", "Ukrainians"→"ukraine",
"Russians"→"russia", "Israelis"→"israel". Adds 5 new tests (63 total).

---------

Co-authored-by: karim <mirakijka@gmail.com>

* chore: strip 61 debug console.log calls from 20 service files (#501)

* chore: strip 61 debug console.log calls from services

Remove development/tracing console.log statements from 20 files.
These add noise to production browser consoles and increase bundle size.

Preserved: all console.error (error handling) and console.warn (warnings).
Preserved: debug-gated logs in runtime.ts (controlled by verbose flag).

Removed: debugInjectTestEvents() from geo-convergence.ts (test-only code).
Removed: logSummary()/logReport() methods that were pure console.log wrappers.

* fix: remove orphaned stubs and remaining debug logs from stripped services

- Remove empty logReport() method and unused startTime variable (parallel-analysis.ts)
- Remove orphaned console.group/console.groupEnd pair (parallel-analysis.ts)
- Remove empty logSignalSummary() export (signal-aggregator.ts)
- Remove logSignalSummary import/call and 3 remaining console.logs (InsightsPanel.ts)
- Remove no-op logDirectFetchBlockedOnce() and dead infrastructure (prediction/index.ts)

* fix: generalize Vercel preview origin regex + include filters in bases cache key (#506)

- api/_api-key.js: preview URL pattern was user-specific (-elie-),
  rejecting other collaborators' Vercel preview deployments.
  Generalized to match any worldmonitor-*.vercel.app origin.

- military-bases.ts: client cache key only checked bbox/zoom, ignoring
  type/kind/country filters. Switching filters without panning returned
  stale results. Unified into single cacheKey string.

* fix(prediction): filter stale/expired markets from Polymarket panel (#507)

Prediction panel was showing expired markets (e.g. "Will US strike Iran
on Feb 9" at 0%). Root causes: no active/archived API filters, no
end_date_min param, no client-side expiry guard, and sub-market selection
picking highest volume before filtering expired ones.

- Add active=true, archived=false, end_date_min API params to all 3
  Gamma API call sites (events, markets, probe)
- Pre-filter sub-markets by closed/expired BEFORE volume selection in
  both fetchPredictions() and fetchCountryMarkets()
- Add defense-in-depth isExpired() client-side filter on final results
- Propagate endDate through all market object paths including sebuf
  fallback
- Show expiry date in PredictionPanel UI with new .prediction-meta
  layout
- Add "closes" i18n key to all 18 locale files
- Add endDate to server handler GammaMarket/GammaEvent interfaces and
  map to proto closesAt field

* fix(relay): guard proxy handlers against ERR_HTTP_HEADERS_SENT crash (#509)

Polymarket and World Bank proxy handlers had unguarded res.writeHead()
calls in error/timeout callbacks that race with the response callback.
When upstream partially responds then times out, both paths write
headers → process crash. Replace 5 raw writeHead+end calls with
safeEnd() which checks res.headersSent before writing.

* feat(breaking-news): add active alert banner with audio for critical/high RSS items (#508)

RSS items classified as critical/high threat now trigger a full-width
breaking news banner with audio alert, auto-dismiss (60s/30s by severity),
visibility-aware timer pause, dedup, and a toggle in the Intelligence
Findings dropdown.

* fix(sentry): filter Android OEM WebView bridge injection errors (#510)

Add ignoreErrors pattern for LIDNotifyId, onWebViewAppeared, and
onGetWiFiBSSID — native bridge functions injected by Lenovo/Huawei
device SDKs into Chrome Mobile WebView. No stack frames in our code.

* chore: add validated telegram channels list (global + ME + Iran + cyber) (#249)

* feat(conflict): add Iran Attacks map layer + strip debug logs (#511)

* chore: strip 61 debug console.log calls from services

Remove development/tracing console.log statements from 20 files.
These add noise to production browser consoles and increase bundle size.

Preserved: all console.error (error handling) and console.warn (warnings).
Preserved: debug-gated logs in runtime.ts (controlled by verbose flag).

Removed: debugInjectTestEvents() from geo-convergence.ts (test-only code).
Removed: logSummary()/logReport() methods that were pure console.log wrappers.

* fix: remove orphaned stubs and remaining debug logs from stripped services

- Remove empty logReport() method and unused startTime variable (parallel-analysis.ts)
- Remove orphaned console.group/console.groupEnd pair (parallel-analysis.ts)
- Remove empty logSignalSummary() export (signal-aggregator.ts)
- Remove logSignalSummary import/call and 3 remaining console.logs (InsightsPanel.ts)
- Remove no-op logDirectFetchBlockedOnce() and dead infrastructure (prediction/index.ts)

* feat(conflict): add Iran Attacks map layer

Adds a new Iran-focused conflict events layer that aggregates real-time
events, geocodes via 40-city lookup table, caches 15min in Redis, and
renders as a toggleable DeckGL ScatterplotLayer with severity coloring.

- New proto + codegen for ListIranEvents RPC
- Server handler with HTML parsing, city geocoding, category mapping
- Frontend service with circuit breaker
- DeckGL ScatterplotLayer with severity-based color/size
- MapPopup with sanitized source links
- iranAttacks toggle across all variants, harnesses, and URL state

* fix: resolve bootstrap 401 and 429 rate limiting on page init (#512)

Same-origin browser requests don't send Origin header (per CORS spec),
causing validateApiKey to reject them. Extract origin from Referer as
fallback. Increase rate limit from 60 to 200 req/min to accommodate
the ~50 requests fired during page initialization.

* fix(relay): prevent Polymarket OOM via request deduplication (#513)

Concurrent Polymarket requests for the same cache key each fired
independent https.get() calls. With 12 categories × multiple clients,
740 requests piled up in 10s, all buffering response bodies → 4.1GB
heap → OOM crash on Railway.

Fix: in-flight promise map deduplicates concurrent requests to the
same cache key. 429/error responses are negative-cached for 30s to
prevent retry storms.

* fix(threat-classifier): add military/conflict keyword gaps and news-to-conflict bridge (#514)

Breaking news headlines like "Israel's strike on Iran" were classified as
info level because the keyword classifier lacked standalone conflict phrases.
Additionally, the conflict instability score depended solely on ACLED data
(1-7 day lag) with no bridge from real-time breaking news.

- Add 3 critical + 18 high contextual military/conflict keywords
- Preserve threat classification on semantically merged clusters
- Add news-derived conflict floor when ACLED/HAPI report zero signal
- Upsert news events by cluster ID to prevent duplicates
- Extract newsEventIndex to module-level Map for serialization safety

* fix(breaking-news): let critical alerts bypass global cooldown and replace HIGH alerts (#516)

Global cooldown (60s) was blocking critical alerts when a less important
HIGH alert fired from an earlier RSS batch. Added priority-aware cooldown
so critical alerts always break through. Banner now auto-dismisses HIGH
alerts when a CRITICAL arrives. Added Iran/strikes keywords to classifier.

* fix(rate-limit): increase sliding window to 300 req/min (#515)

App init fires many concurrent classify-event, summarize-article, and
record-baseline-snapshot calls, exhausting the 200/min limit and causing
429s. Bump to 300 as a temporary measure while client-side batching is
implemented.

* fix(breaking-news): fix fake pubDate fallback and filter noisy think-tank alerts (#517)

Two bugs causing stale CrisisWatch article to fire as breaking alert:
1. Non-standard pubDate format ("Friday, February 27, 2026 - 12:38")
   failed to parse → fallback was `new Date()` (NOW) → day-old articles
   appeared as "just now" and passed recency gate on every fetch
2. Tier 3+ sources (think tanks) firing alerts on keyword-only matches
   like "War" in policy analysis titles — too noisy for breaking alerts

Fix: parsePubDate() handles non-standard formats and falls back to
epoch (not now). Tier 3+ sources require LLM classification to fire.

* fix: make iran-events handler read-only from Redis (#518)

Remove server-side LiveUAMap scraper (blocked by Cloudflare 403 on
Vercel IPs). Handler now reads pre-populated Redis cache pushed from
local browser scraping. Change cache tier from slow to fast to prevent
CDN from serving stale empty responses for 30+ minutes.

* fix(relay): Polymarket circuit breaker + concurrency limiter (OOM fix) (#519)

* fix(rate-limit): increase sliding window to 300 req/min

App init fires many concurrent classify-event, summarize-article, and
record-baseline-snapshot calls, exhausting the 200/min limit and causing
429s. Bump to 300 as a temporary measure while client-side batching is
implemented.

* fix(relay): add Polymarket circuit breaker + concurrency limiter to prevent OOM

Railway relay OOM crash: 280 Polymarket 429 errors in 8s, heap hit 3.7GB.
Multiple unique cache keys bypassed per-key dedup, flooding upstream.

- Circuit breaker: trips after 5 consecutive failures, 60s cooldown
- Concurrent upstream limiter: max 3 simultaneous requests
- Negative cache TTL: 30s → 60s to reduce retry frequency
- Upstream slot freed on response.on('end'), not headers, preventing
  body buffer accumulation past the concurrency cap

* fix(relay): guard against double-finalization on Polymarket timeout

request.destroy() in timeout handler also fires request.on('error'),
causing double decrement of polymarketActiveUpstream (counter goes
negative, disabling concurrency cap) and double circuit breaker trip.

Add finalized guard so decrement + failure accounting happens exactly
once per request regardless of which error path fires first.

* fix(threat-classifier): stagger AI classification requests to avoid Groq 429 (#520)

flushBatch() fired up to 20 classifyEvent RPCs simultaneously via
Promise.all, instantly hitting Groq's ~30 req/min rate limit.

- Sequential execution with 2s min-gap between requests (~28 req/min)
- waitForGap() enforces hard floor + jitter across batch boundaries
- batchInFlight guard prevents concurrent flush loops
- 429/5xx: requeue failed job (with retry cap) + remaining untouched jobs
- Queue cap at 100 items with warn on overflow

* fix(relay): regenerate package-lock.json with telegram dependency

The lockfile was missing resolved entries for the telegram package,
causing Railway to skip installation despite it being in package.json.

* chore: trigger deploy to flush CDN cache for iran-events endpoint

* Revert "fix(relay): regenerate package-lock.json with telegram dependency"

This reverts commit a8d5e1dbbd.

* fix(relay): add POLYMARKET_ENABLED env flag kill switch (#523)

Set POLYMARKET_ENABLED=false on Railway to disable all Polymarket
upstream requests. Returns 503 immediately, preventing OOM crashes.

* fix(breaking-news): fill keyword gaps missing real Iran attack headlines (#521)

* fix(breaking-news): fill keyword gaps that miss real Iran attack headlines

Three root causes for zero alerts during the Iran war:

1. Keyword gaps — top Iran headlines failed classification:
   - "US and Israel attack Iran" → info (no "attack iran" keyword)
   - "attacked Iran" → info (only "attacks iran" existed, plural)
   - "Explosions heard in Tehran" → info (no "explosions" keyword)
   Added: attack iran, attacked iran, attack on iran, attack against iran,
   bombing/bombed iran, war against iran (CRITICAL); explosions,
   launched/launches attacks, retaliatory/preemptive/preventive attack (HIGH)

2. 5-item RSS limit — Al Jazeera's CRITICAL "major combat operations"
   headline was item #7 and never reached the classifier. Increased
   per-feed limit from 5 to 10.

3. False positive — "OpenAI strikes deal with Pentagon" matched HIGH
   keyword "strikes". Added "strikes deal/agreement/partnership" to
   exclusions.

* fix(threat-classifier): prevent Iran compound keyword false positives

"attack iran" as plain substring matched "Iran-backed" and "Iranian"
in headlines about proxy groups, not direct attacks on Iran.

Added TRAILING_BOUNDARY_KEYWORDS set with negative lookahead (?![\w-])
for all Iran compound keywords. This rejects "Iran-backed militias"
and "Iranian targets" while still matching "attack Iran:" and
"attack Iran" at end of string.

Addresses Codex review comment on PR #521.

* fix(relay): regenerate package-lock.json with telegram dependency (#522)

The lockfile was missing resolved entries for the telegram package,
causing Railway to skip installation despite it being in package.json.

* fix(iran): bypass stale CDN cache for iran-events endpoint (#524)

The CDN cached empty {events:[],scrapedAt:0} from the pre-Redis
deployment and Vercel deploy didn't purge all edge nodes. Add ?_v=2
query param to force cache miss until CDN naturally expires the
stale slow-tier entry.

* fix(focal-points): attribute theater military activity to target nations (#525)

The signal aggregator attributed military flights/vessels to the country
they're physically over (point-in-polygon). Aircraft attacking Iran from
the Persian Gulf got attributed to XX/IQ/SA, not IR — so Iran showed
ELEVATED in Focal Points despite being under active attack (CRIT in
Strategic Posture).

Feed theater-level posture data back into the signal aggregator for
target nations (Iran, Taiwan, North Korea, Gaza, Yemen) so they get
credited for military activity in their theater bounding box. Includes
double-count guard to skip if the nation already has signals.

Also fixes stale "sebuf" comment in threat-classifier.

* fix(relay): block rsshub.app requests with 410 Gone (#526)

Stale clients still send RSS requests to rsshub.app (NHK, MOFCOM, MIIT).
These feeds were migrated to Google News RSS but cached PWA clients keep
hitting the relay, which forwards to rsshub.app and gets 403.

- Add explicit blocklist returning 410 Gone before allowlist check
- Remove rsshub.app from all allowlists (relay, edge proxy, vite)
- Remove dead AP News dev proxy target

* feat(map): prioritize Iran Attacks layer (#527)

* feat(map): move Iran Attacks layer to first position and enable by default

Move iranAttacks to the top of the layer toggle list in the full
(geopolitical) variant so it appears first. Enable it by default on
both desktop and mobile during the active conflict.

* feat(map): add Iran Attacks layer support to SVG/mobile map

- Implement setIranEvents() in SVG Map (was no-op)
- Render severity-colored circle markers matching DeckGL layer
- Add iranAttacks to mobile layer toggles (first position)
- Forward setIranEvents to SVG map in MapContainer
- Add IranEventPopupData to PopupData union for click popups
- Add .iran-event-marker CSS with pulse animation
- Add data-layer-hidden-iranAttacks CSS toggle

* fix(geo): expand geo hub index with 60+ missing world locations (#528)

The geo hub index only had ~30 entries, missing all Gulf states (UAE,
Qatar, Bahrain, Kuwait, Oman), Iraq cities, and many world capitals.
News mentioning Abu Dhabi, Dubai, Baghdad, etc. had no lat/lon assigned
so they never appeared on the map.

Added: Gulf capitals (Abu Dhabi, Dubai, Doha, Manama, Kuwait, Muscat),
Iraq (Baghdad, Erbil, Basra), Jordan, Istanbul, Haifa, Dimona, Isfahan,
Kabul, Mumbai, Shanghai, Hong Kong, Singapore, Manila, Jakarta, Bangkok,
Hanoi, Canberra, all major European capitals (Rome, Madrid, Warsaw,
Bucharest, Helsinki, Stockholm, Oslo, Baltics, Athens, Belgrade, Minsk,
Tbilisi, Chisinau, Yerevan, Baku), Americas (Ottawa, Mexico City,
Brasilia, Buenos Aires, Caracas, Bogota, Havana), Africa (Nairobi,
Pretoria, Lagos, Kinshasa, Mogadishu, Tripoli, Tunis, Algiers, Rabat),
conflict zones (Iraq, Kashmir, Golan), chokepoints (Malacca, Panama,
Gibraltar), and US military bases (Ramstein, Incirlik, Diego Garcia,
Guam, Okinawa).

* fix(iran): bust CDN cache to serve updated Gulf-geocoded events (#532)

CDN edge cache was still serving stale 93-event response without
Gulf state coordinates (UAE, Bahrain, Qatar, Kuwait). Bump cache
key from ?_v=2 to ?_v=3 so browsers fetch fresh 100-event data.
Also gitignore internal/ for private tooling scripts.

* fix(alerts): remove SESSION_START gate that blocked pre-existing breaking news (#533)

The isRecent() function used Math.max(now - 15min, SESSION_START) as
recency cutoff. Since SESSION_START = Date.now() at module load, items
published before page load could never trigger alerts — they failed the
SESSION_START gate in the first 15 min, then aged past the 15-min window.

Now uses only the 15-minute recency window. Spam prevention remains via
per-event dedup (30 min), global cooldown (60s), and source tier filter.

* fix(relay): Telegram + OOM + memory cleanup (#531)

* fix(relay): resolve Telegram missing package, OOM crashes, and memory cleanup

- Add `telegram` and `ws` to root dependencies so Railway's `npm install` installs them
- Log V8 heap limit at startup to confirm NODE_OPTIONS is active
- Make MAX_VESSELS/MAX_VESSEL_HISTORY env-configurable (default 20k, down from 50k)
- Add permanent latch to skip Telegram import retries when package is missing
- Raise memory cleanup threshold from 450MB to 2GB (env-configurable)
- Clear all caches (RSS, Polymarket, WorldBank) during emergency cleanup

* fix(relay): treat blank env vars as unset in safeInt

Number('') === 0 passes isFinite, silently clamping caps to 1000
instead of using the 20000 default. Guard empty/null before parsing.

* fix(live-news): replace 7 stale YouTube fallback video IDs (#535)

Validated all 23 YouTube fallbackVideoIds via oEmbed API and all 9
HLS URLs. Found 5 broken IDs (403 embed-restricted or 404 deleted)
plus 2 previously identified stale IDs:

- Fox News: QaftgYkG-ek → ZvdiJUYGBis
- Sky News Arabia: MN50dHFHMKE → U--OjmpjF5o
- RTVE 24H: 7_srED6k0bE → -7GEFgUKilA
- CNN Brasil: 1kWRw-DA6Ns → 6ZkOlaGfxq4
- C5N: NdQSOItOQ5Y → SF06Qy1Ct6Y
- TBS NEWS DIG: ohI356mwBp8 → Anr15FA9OCI
- TRT World: CV5Fooi8WDI → ABfFhWzWs0s

All 9 HLS URLs validated OK. 16 remaining YouTube IDs validated OK.

* fix(relay): fix telegram ESM import path and broaden latch regex

- `import('telegram/sessions')` fails with "Directory import is not
  supported resolving ES modules" — use explicit `telegram/sessions/index.js`
- Broaden permanent-disable latch to also catch "Directory import" errors

* fix(ui): move download banner to bottom-right (#536)

* fix(alerts): remove SESSION_START gate that blocked pre-existing breaking news

The isRecent() function used Math.max(now - 15min, SESSION_START) as
recency cutoff. Since SESSION_START = Date.now() at module load, items
published before page load could never trigger alerts — they failed the
SESSION_START gate in the first 15 min, then aged past the 15-min window.

Now uses only the 15-minute recency window. Spam prevention remains via
per-event dedup (30 min), global cooldown (60s), and source tier filter.

* fix(ui): move download banner to bottom-right of screen

Repositioned from top-right (overlapping content) to bottom-right.
Dismissal already persists via localStorage. Added TODO for header
download link.

* Revert "fix(relay): fix telegram ESM import path and broaden latch regex"

This reverts commit 1f2f0175ab.

* Revert "Revert "fix(relay): fix telegram ESM import path and broaden latch regex"" (#537)

This reverts commit ad41a2e2d245850fd4b699af2adbe53acca80325.

* feat: add day/night solar terminator overlay to map (#529)

* Trigger redeploy with preview env vars

* Trigger deployment

* chore: trigger redeploy for PR #41

* chore: trigger Vercel redeploy (edge function transient failure)

* chore: retrigger Vercel deploy

* feat: add Nigeria feeds and Greek locale feeds (#271)

- Add 5 Nigeria news sources to Africa section (Premium Times, Vanguard,
  Channels TV, Daily Trust, ThisDay)
- Add 5 Greek feeds with lang: 'el' for locale-aware filtering
  (Kathimerini, Naftemporiki, in.gr, iefimerida, Proto Thema)
- Add source tiers for all new outlets
- Allowlist 8 new domains in RSS proxy

* fix: enforce military bbox filtering and add behavioral cache tests (#284)

* fix: add request coalescing to Redis cache layer

Concurrent cache misses for the same key now share a single upstream
fetch instead of each triggering redundant API calls. This eliminates
duplicate work within Edge Function invocations under burst traffic.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: reduce AIS polling frequency from 10s to 30s

Vessel positions do not change meaningfully in 10 seconds at sea.
Reduces Railway relay requests by 66% with negligible UX impact.
Stale threshold bumped to 45s to match the new interval.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: quantize military flights bbox cache keys to 1-degree grid

Precise bounding box coordinates caused near-zero cache hit rate since
every map pan/zoom produced a unique key. Snapping to a 1-degree grid
lets nearby viewports share cache entries, dramatically reducing
redundant OpenSky API calls.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: parallelize ETF chart fetches instead of sequential await loop

The loop awaited each ETF chart fetch individually, blocking on every
Yahoo gate delay. Using Promise.allSettled lets all 10 fetches queue
concurrently through the Yahoo gate, cutting wall time from ~12s to ~6s.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add Redis pipeline batch GET to reduce round-trips

Add getCachedJsonBatch() using the Upstash pipeline API to fetch
multiple keys in a single HTTP call. Refactor aircraft details batch
handler from 20 sequential GETs to 1 pipelined request.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add structural tests for Redis caching optimizations

18 tests covering: cachedFetchJson request coalescing (in-flight dedup,
cache-before-fetch ordering, cleanup), getCachedJsonBatch pipeline API,
aircraft batch handler pipeline usage, bbox grid quantization (1-degree
step, expanded fetch bbox), and ETF parallel fetch.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: enforce military bbox contract and add behavioral cache tests

---------

Co-authored-by: Elias El Khoury <efk@anghami.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add User-Agent and Cloudflare 403 detection to all secret validation probes (#296)

Sidecar validation probes were missing User-Agent headers, causing
Cloudflare-fronted APIs (e.g. Wingbits) to return 403 which was
incorrectly treated as an auth rejection. Added CHROME_UA to all 13
probes and isCloudflare403() helper to soft-pass CDN blocks.

* fix: open external links in system browser on Tauri desktop (#297)

Tauri WKWebView/WebView2 traps target="_blank" navigation, so news
links and other external URLs silently fail to open. Added a global
capture-phase click interceptor that routes cross-origin links through
the existing open_url Tauri command, falling back to window.open.

* fix: add Greek flag mapping to language selector (#307)

* fix: add missing country brief i18n keys and export PDF option (#308)

- Add levels, trends, fallback keys to top-level countryBrief in en/el/th/vi
  locales (fixes raw key display in intelligence brief and header badge)
- Add Export PDF option to country brief dropdown using scoped print dialog
- Add exportPdf i18n key to all 17 locale files

* feat: add day/night solar terminator overlay to map

Add a real-time day/night overlay layer using deck.gl PolygonLayer that
renders the solar terminator (boundary between day and night zones).
The overlay uses astronomical formulas (Meeus) to compute the subsolar
point and trace the terminator line at 1° resolution.

- New toggleable "Day/Night" layer in all 3 variants (full/tech/finance)
- Theme-aware styling (lighter fill on light theme, darker on dark)
- Auto-refresh every 5 minutes with conditional timer (only runs when
  layer is enabled, pauses when render is paused)
- Cached polygon computation to avoid recomputing on every render
- i18n translations for all 17 locales
- Updated documentation with new layer entry

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address review feedback — equinox terminator + locale indentation

- Replace safeTanDecl epsilon clamp with proper equinox handling:
  when |tanDecl| < 1e-6, draw terminator as vertical great circle
  through the poles (subsolar meridian ±90°) instead of clamping
- Fix JSON indentation in all 17 locale files: dayNight and
  tradeRoutes keys were left-aligned instead of matching 8-space
  indentation of surrounding keys

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
Co-authored-by: Elias El Khoury <efk@anghami.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>

* fix(relay): auto-reconnect on Telegram AUTH_KEY_DUPLICATED and fix IranIntl handle (#539)

- On AUTH_KEY_DUPLICATED (406), disconnect client and set to null so
  next poll cycle reconnects fresh — self-heals after competing client dies
- Fix IranIntl → iranintltv (correct Telegram channel handle)

* fix(live-news): add fallback video ID for LiveNOW from FOX channel (#538)

The livenow-fox channel had no fallbackVideoId, relying solely on
YouTube handle lookup which fails intermittently. Added ZvdiJUYGBis
(confirmed live stream) as fallback.

* fix(iran): bump CDN cache-bust to v4 for fresh event data (#544)

100 new events pushed to Redis covering active Iran-Israel-US
conflict theater including Gulf states (UAE, Bahrain, Qatar,
Kuwait, Jordan). Bump ?_v=3 to ?_v=4 to bypass stale CDN.

* fix(telegram): fix ESM import path in session-auth script (#542)

telegram/sessions → telegram/sessions/index.js (same fix as relay)

* fix(telegram): latch AUTH_KEY_DUPLICATED to stop retry spam (#543)

AUTH_KEY_DUPLICATED is permanent — the session string is invalidated
and no amount of retrying will fix it. Previously the relay retried
every 60s, spamming logs. Now it logs a clear error message with
instructions to regenerate the session and stops retrying.

Renamed telegramImportFailed → telegramPermanentlyDisabled to cover
both import failures and auth failures under one latch.

* fix(live-news): fix broken Europe channel handles + add fallback video IDs (#541)

* fix(live-news): fix broken Europe channel handles + add fallback video IDs

- Fix France 24 English handle: @FRANCE24English (404) → @France24_en
- Fix WELT handle: @WELTNachrichtensender (hijacked to "Movie Glow") → @WELTVideoTV
- Add fallbackVideoId for BBC News, France 24 EN, TRT Haber, NTV Turkey,
  CNN TURK, TVP Info, Telewizja Republika (verified via Playwright)
- Update stale fallback IDs for Fox News, RTVE, CNN Brasil, C5N, TBS News,
  Sky News Arabia, TRT World

* fix(live-news): update CBS News fallback video ID

* fix(live-news): update Newsmax fallback video ID

* fix(live-news): add NBC News fallback video ID

* fix(live-news): full channel audit — fix 10 broken handles + update 8 stale fallbacks

Broken handles fixed:
- Bloomberg: @Bloomberg (404) → @markets
- WION: @WIONews (wrong channel "Write It Out") → @WION
- CTI News: @CtiTv (404) → @中天新聞CtiNews
- VTC NOW: @VTCNOW (404) → @VTCNowOfficial
- Record News: @recordnewsoficial (404) → @RecordNews
- T13: @T13 (404) → @Teletrece
- Channels TV: @channelstv (404) → @ChannelsTelevision
- KTN News: @KTNNewsKE (404) → @ktnnews_kenya
- eNCA: @enewschannel (404) → @eNCA
- SABC News: @SABCNews (404) → @SABCDigitalNews

Stale fallback video IDs refreshed:
- Sky News, NASA, CBC News, CNN Brasil, C5N, TBS NEWS DIG,
  Sky News Arabia, TRT World

* feat(oref): add OREF sirens panel with Hebrew-to-English translation (#545)

Add real-time Israel Home Front Command (OREF) siren alerts panel:
- Edge Function proxy at api/oref-alerts.js
- OrefSirensPanel component with live/history views
- oref-alerts service with 10s polling and update callbacks
- Hebrew→English translation via existing translateText() LLM chain
  with 3-layer caching (in-memory Map → server Redis → circuit breaker)
- i18n strings for all 23 locales
- Panel registration, data-loader integration, CSS styles

* fix(relay): use execFileSync for OREF curl to avoid shell injection (#546)

Proxy credentials with special characters (semicolons, dollar signs)
were interpolated into a shell command via execSync. Switch to
execFileSync which passes args directly without shell parsing.

* gave the user freedom to resize panels "fixes issue #426" (#489)

* gave the user freedom to resize panles

* feat(panels): add horizontal resize with col-span persistence

* feat(cii): integrate Iran strike events into CII scoring, country brief & timeline (#547)

Iran had ~100 geolocated strike events but the CII was unaware of them:
conflict score stuck at 70 (ACLED only), no strike chip in Active Signals,
timeline conflict lane empty, intelligence brief silent on strikes.

Changes:
- Add strikes[] to CountryData and ingestStrikesForCII() with geo-lookup
  fallback (bounding boxes when GeoJSON not yet loaded)
- Boost CII conflict score with 7-day freshness window
  (min(50, count*3 + highSev*5))
- Cache iranEvents in IntelligenceCache, preserve across refresh cycles
- Wire data loading: always load Iran events (not gated by map layer),
  ingest into CII, trigger panel refresh
- Add activeStrikes to CountryBriefSignals with geo-lookup counting
- Render strike chip in Active Signals and include in fallback brief
- Feed strike events into 7-day timeline (conflict lane)
- Add structured strikeCount/highSeverityStrikeCount fields to GeoSignal
  (replaces fragile regex parsing in focal-point-detector)
- Add active_strike signal type to InsightsPanel focal points
- Add bounding-box fallback to signal aggregator for conflict events
- Add i18n keys for activeStrikes

* fix(alerts): add compound escalation for military action + geopolitical target (#548)

Keyword matching was too rigid — "attacks on iran" didn't match CRITICAL
because only "attack on iran" (singular) existed. Headlines like
"strikes by US and Israel on Iran" also missed because words weren't
adjacent.

Added compound escalation: if a HIGH military/conflict keyword matches
AND the headline mentions a critical geopolitical target (Iran, Russia,
China, Taiwan, NATO, US forces), escalate to CRITICAL. Also added
missing Iran keyword variants (plural forms, "Iran retaliates/strikes").

* feat(conflict): enhance Iran events popup with severity badge and related events (#549)

Rewrite the Iran events popup to follow the established popup pattern
(conflict/protest) with severity-colored header, badge, close button,
stat rows, and source link using CSS classes instead of inline styles.

- Add normalizeSeverity helper (clamps unknown values to 'low')
- Show related events from same location (normalized matching, max 5)
- Add IranEventPopupData to PopupData union (removes unsafe double cast)
- Add iranEvent header CSS with severity border-left colors
- Add i18n keys for en/ar/fr

* feat(telegram): add Telegram Intel panel (#550)

* feat(telegram): add Telegram Intel panel consuming relay feed

- Service layer: fetchTelegramFeed() with 30s cache, types matching relay shape
- Panel component: topic filter tabs, safe DOM rendering via h()+replaceChildren()
- DataLoader + RefreshScheduler pattern (60s interval, hidden-tab aware)
- Handles enabled=false and empty states from relay
- CSS following existing gdelt-intel pattern
- Panel title localized across all 18 locales

* fix(i18n): add components.telegramIntel translations to 10 remaining locales

* feat(live-news): add residential proxy + gzip decompression for YouTube detection (#551)

YouTube blocks Vercel datacenter IPs — returns HTML without videoDetails/isLive
data. Switch from edge runtime to Node.js serverless to enable HTTP CONNECT
tunnel proxy via YOUTUBE_PROXY_URL env var. Add zlib decompression for gzip
responses (YouTube returns empty body without Accept-Encoding header).

Also adds missing fallback video IDs for WELT, KTN News, CNA NewsAsia,
and updates TBS NEWS DIG fallback.

* debug(live-news): add debug param to diagnose proxy env var on Vercel

* fix(live-news): set explicit runtime: 'nodejs' for proxy support

Vercel defaults to edge runtime when not specified. node:http/https/zlib
imports are unavailable in edge — causing FUNCTION_INVOCATION_FAILED.
Remove debug param added in previous commit.

* fix(live-news): lazy-load node modules + proxy fallback to direct fetch

Top-level import of node:http/https/zlib crashes if Vercel bundles
for edge despite runtime: 'nodejs' config. Use dynamic import() to
lazy-load at call time. Also add try/catch around proxy so it falls
back to direct fetch if proxy connection fails.

* feat(aviation): integrate AviationStack API for non-US airport delays (#552)

Replace 100% simulated delay data for international airports with real
flight data from AviationStack API. Add 28 Middle East/conflict-zone
airports (Iran, Iraq, Lebanon, Syria, Yemen, Pakistan, Libya, Sudan).

Key changes:
- AviationStack integration with bounded concurrency (5 parallel),
  rotating batch (20 airports/cycle), and 20s deadline
- Redis SETNX lock prevents cross-isolate cache stampede on expiry
- Split FAA/intl caches (both 30min TTL) with isolated error handling
- Fix severity colors (was checking 'GS'/'GDP', now minor/moderate/major/severe)
- Fix tooltip (was obj.airport, now obj.name + obj.iata)
- Add FLIGHT_DELAY_TYPE_CLOSURE for airport/airspace closures
- Add closure i18n key across all 18 locales
- Graceful fallback: no API key → simulation; API failure → simulation

* feat(live-news): move YouTube proxy scraping to Railway relay

Vercel serverless cannot use node:http/https for HTTP CONNECT proxy
tunnels. Move the residential proxy YouTube scraping to the Railway
relay (ais-relay.cjs) which has full Node.js access.

- Add /youtube-live route to relay with proxy + direct fetch fallback
- Add 5-min in-memory cache for channel lookups, 1hr for oembed
- Revert Vercel api/youtube/live.js to edge runtime — now proxies to
  Railway first, falls back to direct scrape

* feat(settings): add AVIATIONSTACK_API to desktop settings page (#553)

Register the key in Rust keychain (SUPPORTED_SECRET_KEYS), frontend
RuntimeSecretKey type, feature toggle, and settings UI under
"Tracking & Sensing" category.

* fix(live-news): use correct relay auth header for YouTube proxy (#554)

Edge function was sending X-Relay-Auth header with RELAY_AUTH_TOKEN env
var, but the Railway relay expects x-relay-key header validated against
RELAY_SHARED_SECRET. This mismatch caused the relay to reject requests
from Vercel, falling back to direct YouTube scrape (which fails from
datacenter IPs for many channels).

* fix(live-news): align YouTube edge function with relay auth pattern (#555)

Use same getRelayBaseUrl/getRelayHeaders as other edge functions:
- WS_RELAY_URL env var instead of VITE_WS_API_URL
- RELAY_SHARED_SECRET + RELAY_AUTH_HEADER for flexible auth
- Dual x-relay-key + Authorization headers

* fix(i18n): rename OREF Sirens panel to Israel Sirens (#556)

Remove internal implementation references (OREF, proxy relay, oref.org.il)
from all user-facing strings across 18 locales and panel config.

* fix(live-news): annotate empty catches and sanitize error output (#560)

- Add context comments to empty catch blocks for debuggability
- Replace error.message leak with generic client-safe message

* fix(sentry): add noise filters and fix beforeSend null-filename leak (#561)

- Add 8 new ignoreErrors patterns: signal timeout, premium gate,
  hybridExecute/mag/webkit bridge injections, postMessage null,
  NotSupportedError, appendChild injection, luma assignment
- Fix LIDNotify regex to match both LIDNotify and LIDNotifyId
- Fix beforeSend: strip null/anonymous filename frames so deck.gl
  TypeErrors (28 events, 8 users) are properly suppressed

* feat(cii): wire OREF sirens into CII score & country brief (#559)

* feat(cii): wire OREF sirens into CII score and country brief

Active OREF sirens now boost Israel's CII score through two channels:
- Conflict component: +25 base + min(25, alertCount*5) for active sirens
- Blended score: +15 for active sirens, +5/+10 for 24h history thresholds

Country brief for Israel shows a siren signal chip when alerts are active.

* refactor(cii): extract getOrefBlendBoost helper to DRY scoring paths

* fix(relay): add graceful shutdown + poll concurrency guard for Telegram (#562)

- SIGTERM/SIGINT handler disconnects Telegram client before container dies
- telegramPollInFlight guard prevents overlapping poll cycles
- Mid-poll AUTH_KEY_DUPLICATED now permanently disables (was reconnect loop)

* fix(aviation): query all airports instead of rotating batch (#557)

* fix(aviation): query all airports instead of rotating batch of 20

The rotating batch (20 airports/cycle) caused major airports like DXB
(52% cancellations) to be missed entirely for multiple cache cycles.
With a paid AviationStack plan, query all ~90 non-US airports per
refresh with concurrency 10 and 50s deadline (~9 chunks × 5s = 45s).

* feat(cii): feed airport disruptions into CII and country brief

Major/severe airport delays and closures now boost the CII security
score and appear as signal chips in country briefs. Only major+
severity alerts are ingested to avoid noise from minor delays.

- Add aviationDisruptions to CountryData and ingestAviationForCII()
- Boost security score: closure +20, severe +15, major +10, moderate +5
- Store flight delays in intelligenceCache for country brief access
- Add aviation disruptions chip in country brief signals grid

* fix(relay): replace smart quotes crashing relay on startup (#563)

* fix(relay): replace Unicode smart quotes crashing Node.js CJS parser

* fix(relay): await Telegram disconnect + guard startup poll

* fix(cii): resolve Gulf country strike misattribution via multi-match bbox disambiguation (#564)

Dubai/Doha/Bahrain/Kuwait coordinates matched Iran's bounding box first
due to iteration order. Now collects ALL matching bboxes, disambiguates
via isCoordinateInCountry() geometry, and falls back to smallest-area bbox.

- Add BH, QA, KW, JO, OM to bounds tables (previously missing entirely)
- Extract ME_STRIKE_BOUNDS + resolveCountryFromBounds() into country-geometry.ts
- All 4 consumer files use shared constant (single source of truth)
- Bump CDN cache-bust param for iran-events endpoint

* fix(relay): upstreamWs → upstreamSocket in graceful shutdown (#565)

* fix(relay): install curl in Railway container for OREF polling (#567)

* fix(relay): increase Polymarket cache TTL to 10 minutes (#568)

* fix(relay): increase Polymarket cache TTL to 10 minutes

All requests were MISS with 2-min TTL under concurrent load.
Bump to 10-min cache and 5-min negative cache to reduce upstream pressure.

* fix(relay): normalize Polymarket cache key from canonical params

Raw url.search as cache key meant ?tag=fed&endpoint=events and
?endpoint=events&tag_slug=fed produced different keys for the same
upstream request — defeating both cache and inflight dedup, causing
121 MISS entries in 3 seconds.

Build cache key from parsed canonical params (endpoint + sorted
query string) so all equivalent requests share one cache entry.

* feat(webcams): add Iran tab to live webcams panel (#569)

Add dedicated Iran region tab as the first/default tab with 4 feeds:
Tehran, Middle East overview, Tehran (alt angle), and Jerusalem.

* fix(relay): replace nixpacks.toml with railpack.json for curl (#571)

Railway uses Railpack (not Nixpacks). nixpacks.toml in scripts/ was
silently skipped. Use railpack.json at repo root with deploy.aptPackages
to install curl at runtime for OREF polling.

* fix(webcams): replace duplicate Tehran feed with Tel Aviv, rename Iran tab (#572)

- Remove duplicate iran-tehran2 feed (same channel/video as iran-tehran)
- Remove iran-mideast feed
- Add Tel Aviv feed (-VLcYT5QBrY) to Iran Attacks tab
- Rename tab label from "IRAN" to "IRAN ATTACKS" across all 18 locales

* feat(scripts): add Iran events seed script and latest data (#575)

Add seed-iran-events.mjs for importing Iran conflict events into Redis
(conflict:iran-events:v1). Includes geocoding by location keywords and
category-to-severity mapping. Data file contains 100 events from
2026-02-28.

* fix(relay): add timeouts and logging to Telegram poll loop (#578)

GramJS getEntity/getMessages have no built-in timeout. When the first
channel hangs (FLOOD_WAIT, MTProto stall), telegramPollInFlight stays
true forever, blocking all future polls — zero messages collected, zero
errors logged, frontend shows "No messages available".

- Add 15s per-channel timeout on getEntity + getMessages calls
- Add 3-min overall poll cycle timeout
- Force-clear stuck in-flight flag after 3.5 minutes
- Detect FLOOD_WAIT errors and break loop early
- Log per-cycle summary: channels polled, new msgs, errors, duration
- Track media-only messages separately (no text → not a bug)
- Expose lastError, pollInFlight, pollInFlightSince on /status endpoint

* feat(cii): hook security advisories into CII scoring & country briefs (#579)

Travel advisories (Do Not Travel, Reconsider, Caution) from US, AU, UK,
NZ now act as a floor and boost on CII scores. Do Not Travel guarantees
a minimum score of 60 (elevated), Reconsider floors at 50. Multi-source
corroboration (3+ govts) adds +5 bonus.

Advisory chips appear in country brief signal grid with level-appropriate
styling, and advisory context is passed to AI brief generation.

- Extract target country from advisory titles via embassy feed tags and
  country name matching
- Add advisoryMaxLevel/advisoryCount/advisorySources to CII CountryData
- Wire ingestAdvisoriesForCII into data loader pipeline
- Add travelAdvisories/travelAdvisoryMaxLevel to CountryBriefSignals
- Render advisory signal chips in CountryBriefPage

* fix(sentry): guard setView against invalid preset + filter translateNotifyError (#580)

- DeckGLMap.setView(): early-return if VIEW_PRESETS[view] is undefined,
  preventing TypeError on 'longitude' when select value is invalid
- Add ignoreErrors pattern for Google Translate widget crash

* feat(relay): bootstrap OREF 24h history on startup (#582)

* fix(relay): improve OREF curl error logging with stderr capture

-s flag silenced curl errors. Add -S to show errors, capture stderr
via stdio pipes, and log curl's actual error message instead of
generic "Command failed" from execFileSync.

* feat(relay): bootstrap OREF 24h history on startup and add missing headers

- Fetch AlertsHistory.json once on startup to populate orefState.history
  immediately instead of starting empty
- Add X-Requested-With: XMLHttpRequest header required by Akamai WAF
- Add IST→UTC date converter handling DST ambiguity
- Redact proxy credentials from error logs and client responses
- Fix historyCount24h to count individual alert records, not snapshots
- Guard historyCount24h reducer for both array and string data shapes
- Add input validation to orefDateToUTC for malformed dates

* feat(aviation): add comprehensive logging to flight delay pipeline (#581)

Adds structured [Aviation] logging throughout the handler chain:
- Handler: timing, FAA/intl alert counts
- fetchIntlWithLock: cache hit/miss, lock status, fallback triggers
- fetchAviationStackDelays: airport count, deadline hits, success/fail stats
- fetchSingleAirport: per-airport flight stats, API errors, severity
- Annotate empty catches with error context

* feat: add GPS/GNSS jamming map layer + CII integration (#570)

* feat: add GPS/GNSS jamming data ingestion from gpsjam.org

- scripts/fetch-gpsjam.mjs: standalone fetcher that downloads daily H3
  hex data, filters medium/high interference, converts to lat/lon via
  h3-js, and writes JSON. Can be run on cron.
- api/gpsjam.js: Vercel Edge Function that proxies gpsjam.org data with
  1hr cache, returns medium/high hexes for frontend consumption.
- src/services/gps-interference.ts: frontend service that fetches from
  the Edge API, converts H3→lat/lon, and classifies by conflict region.
- h3-js added as dependency for hex→coordinate conversion.

* feat: add GPS jamming map layer, CII integration, and country brief signals

Wire gpsjam.org data into map visualization, instability scoring, and
country intelligence. ScatterplotLayer renders high (red) and medium
(orange) interference hexes. CII security score incorporates jamming
counts per country via h3→country geocoding with cache. Country briefs
show jamming zone chip. Full i18n across 18 locales including popup
labels. Data loads with intelligence signals cycle (15min), gated by
1hr client-side cache.

* feat(aviation): add NOTAM closure detection via ICAO API (#583)

* feat(aviation): add NOTAM closure detection via ICAO API

Adds international airport closure detection via ICAO NOTAMs:
- New fetchNotamClosures() queries ICAO realtime-notams endpoint
- Detects closures via Q-codes (FA/AH/AL/AW/AC/AM) and text patterns
- Batches airports in groups of 20 per API call
- 4-hour cache TTL via cachedFetchJson (stampede-safe)
- NOTAM closures override existing AviationStack alerts for same airport
- Graceful: no ICAO_API_KEY env var = silently skipped

To activate: set ICAO_API_KEY env var (register at dataservices.icao.int)

* feat(settings): add ICAO_API_KEY to desktop app settings

Adds ICAO NOTAM API key to the desktop settings UI:
- Rust: SUPPORTED_SECRET_KEYS [23→24]
- TypeScript: RuntimeSecretKey + RuntimeFeatureId unions
- Feature definition: 'icaoNotams' in Tracking & Sensing category
- Settings UI: label, signup URL, analytics name

* feat(aviation): limit NOTAM queries to MENA airports only

Per user request, ICAO NOTAM closure detection is scoped to
Middle East airports only (region='mena', ~35 airports).
This reduces API calls (2 batches vs 5) and focuses on the
region where closures are most relevant.

* fix(aviation): align NOTAM cache TTL to 30 min (matching FAA/intl)

* feat(risk): wire theater posture + breaking news into strategic risk score (#584)

The composite score ignored theater military buildup and breaking news
alerts, causing misleadingly low scores during active military events.

- Add theater boost from raw asset counts + strike capability (capped +25)
- Add breaking news severity boost (critical=15, high=8, capped +15)
- Listen for wm:breaking-news events with 30-min TTL and auto-expiry
- Read cached theater postures via getCachedPosture() with stale discount
- Lower trend threshold from ±5 to ±3 for faster escalation detection
- Cleanup listeners and timers in destroy()

* fix(relay): delay Telegram connect 60s on startup to prevent AUTH_KEY_DUPLICATED (#587)

Railway zero-downtime deploys start the new container before the old one
receives SIGTERM. Both containers connect with the same session string
simultaneously, triggering Telegram's AUTH_KEY_DUPLICATED which permanently
invalidates the session. A 60s startup delay gives the old container time
to disconnect gracefully. Configurable via TELEGRAM_STARTUP_DELAY_MS env.

* feat(feeds): add RT (Russia Today) RSS feeds (#585)

- Add RT main feed (rt.com/rss/) and RT Russia desk (rt.com/rss/russia/)
- Add to SOURCE_TIERS (tier 3), SOURCE_TYPES (wire), SOURCE_CREDIBILITY
- Add rt.com to rss-proxy ALLOWED_DOMAINS

* feat(live-news): add RT channel via HLS + enable HLS on web (#586)

- Add RT (rt.com) as optional channel in Europe region
- HLS stream: rt-glb.rttv.com/dvr/rtnews/playlist.m3u8 (CORS: *)
- Enable native <video> HLS playback on web (was desktop-only)
- Channels in DIRECT_HLS_MAP with CORS headers now work everywhere

* fix(telegram): add missing relay auth headers to telegram-feed edge function (#590)

The edge function was the only relay proxy missing RELAY_SHARED_SECRET
auth headers. The relay returns 401 for all non-public routes, so the
panel always received an error response → "No messages available".

* fix(aviation): replace broken lock mechanism with direct cache, add cancellation tiers (#591)

The Redis SETNX lock for AviationStack had a 3s wait but the API takes
~25s for 50 airports. Every lock-loser fell back to simulation — meaning
AviationStack data was never served despite the API key being configured.

Changes:
- Remove fetchIntlWithLock/tryAcquireLock, use getCachedJson/setCachedJson
  with conditional TTL (30min real data, 5min simulation fallback)
- Add cancellation severity tiers: ≥20% moderate, ≥10% minor (DXB at 32%
  cancelled was previously dropped as null)
- Bump cache key to v2 to invalidate stale simulation data
- Add HTML content-type detection for NOTAM API (ICAO CloudFront returns
  HTML challenge pages from Vercel datacenter IPs)

* fix(relay): stop Polymarket cache stampede from concurrent limit + CDN bypass (#592)

Three issues caused continuous MISS every 5 seconds:

1. Concurrent limit rejection poisoned cache: 11 tags fire via Promise.all
   but POLYMARKET_MAX_CONCURRENT=3, so 8 tags got negative-cached with
   empty [] (5 min TTL). Those 8 tags NEVER got positive cache because
   they were always throttled. Fix: replace reject-with-negative-cache
   with a proper queue — excess requests wait for a slot instead of
   being silently rejected.

2. Cache key fragmentation: fetchPredictions(limit=20) and
   fetchCountryMarkets(limit=30) created separate cache entries for the
   same tag. Fix: normalize to canonical limit=50 upstream, cache key
   is shared regardless of caller's requested limit.

3. CDN bypass: end_date_min timestamp in query string made every URL
   unique, preventing Vercel CDN caching entirely. Fix: strip
   end_date_min, active, archived from proxy params (relay ignores them
   anyway).

* fix(polymarket): add queue backpressure and response limit slicing (#593)

- Add POLYMARKET_MAX_QUEUED=20 cap to prevent unbounded queue growth
  under sustained load (rejects with negative cache when full)
- Use requestedLimit to slice cached responses — callers requesting
  limit=20 now get 20 items instead of the full 50-item upstream payload
- Hoist PROXY_STRIP_KEYS Set to module level (avoids per-call allocation)

* fix(webcams): add 4th Iran Attacks feed to fill 2x2 grid (#601)

Add Middle East multi-cam (4E-iFtUM2kk) as 4th Iran region feed.
Previously only 3 feeds for a 4-cell grid, leaving one cell black.

* fix(aviation): route NOTAM through relay + improve intl logging (#599)

Root cause: ICAO NOTAM API times out from Vercel edge (>10s).
AviationStack alerts indistinguishable from simulation in logs.

Changes:
- Add /notam proxy endpoint to Railway relay (25s timeout, 30min cache)
- Route fetchNotamClosures through relay when WS_RELAY_URL is set
- Fall back to direct ICAO call (20s timeout) when no relay
- Log cache hits with real vs simulated alert counts
- Send all MENA airports in single NOTAM request (was batched by 20)

Requires: ICAO_API_KEY env var on Railway relay

* chore(telegram): update channel list — remove nexta_live, air_alert_ua; add wfwitness (#600)

* fix(sentry): guard YT player methods + filter GM/InvalidState noise (#602)

- Guard loadVideoById/cueVideoById with typeof check in LiveNewsPanel
  (race condition: YT.Player object exists before onReady fires)
- Add ignoreErrors for GM_getValue (Greasemonkey extension) and
  InvalidStateError (IndexedDB/DOM state errors from browser internals)

* fix(aviation): always show all monitored airports on flight delays map (#603)

* fix(aviation): always show MENA airports on map regardless of delay status

MENA airports only appeared when they had active delays/closures.
GCC airports like DOH, AUH, RUH were invisible during normal operations.
Now fills in "normal operations" entries for all MENA airports without
alerts so they always render as gray dots on the flight delays layer.

* fix(aviation): show all monitored airports globally, not just MENA

Extend normal-operations fill to all 128 monitored airports worldwide,
not just the 35 MENA airports. Any airport without active delays now
appears as a gray dot on the flight delays map layer.

* fix(webcams): fix broken live news channels — eNCA handle, remove VTC NOW, fix CTI News (#604)

- eNCA: fix YouTube handle from @eNCA to @encanews
- VTC NOW: remove — VTC shut down Jan 2025 (Vietnam govt restructuring)
- CTI News: remove stale fallbackVideoId and useFallbackOnly, let auto-detect work

* docs(readme): comprehensive update for Telegram, OREF, GPS jamming, airports, and more (#606)

Reflects 30+ recent commits with updates across the README:
- Update data layer counts (36 → 40+), webcam counts (19 → 22), data sources (16 → 28+)
- Add 6 new How It Works sections: Telegram OSINT, OREF Rocket Alerts, GPS/GNSS
  Interference, Security Advisories, Airport Delays/NOTAMs, Strategic Risk Score
- Expand Railway relay section with service table (AIS, OpenSky, Telegram, OREF,
  Polymarket, NOTAM) and update architecture ASCII diagram
- Add Iran/Attacks webcam tab, HLS native streaming details, RT coverage
- Add Algorithmic Design Decisions section (log vs linear scoring, Welford's,
  H3 hex grids, cosine-lat correction, negative caching)
- Add 2 new architecture principles (graceful degradation, multi-source corroboration)
- Add 8 new roadmap items for recently shipped features
- Update Tech Stack table with Telegram, OREF, gpsjam.org in Geopolitical APIs

* chore: bump version to 2.5.21 (#605)

* fix(aviation): invalidate stale IndexedDB cache + reduce CDN TTL (#607)

Bump circuit breaker name from 'FAA Flight Delays' to 'Flight Delays v2'
to force all clients to discard stale IndexedDB entries that predate
PR #603 (which added normal-ops airport fill). Also downgrade CDN cache
tier from 'slow' (15 min) to 'medium' (5 min) since airport status
changes more frequently than other slow-tier endpoints.

* fix(relay): increase OREF curl maxBuffer to prevent ENOBUFS (#609)

* fix(relay): increase OREF curl maxBuffer to 10MB to prevent ENOBUFS

AlertsHistory.json response exceeds execFileSync default 1MB buffer,
causing spawnSync ENOBUFS on Railway container at startup.

* fix(relay): use curl -o tmpfile for OREF history instead of stdout buffer

Large AlertsHistory.json overflows execFileSync stdout buffer (ENOBUFS).
Now writes to temp file via curl -o, reads with fs.readFileSync, cleans up.
Live alerts (tiny payload) still use stdout path.

* fix: RT channel HLS-only recovery, test shim, and LiveNOW fallback (#610)

- Remove useFallbackOnly from RT channel — RT is HLS-only (banned from
  YouTube), so the flag was causing undefined videoId on HLS failure
  instead of graceful offline state
- Add response-headers shim to redis-caching test so military flights
  bbox tests can import list-military-flights.ts
- Restore LiveNOW from FOX fallbackVideoId (removed in channel audit)

* fix(docs): add blank lines after CHANGELOG headings for markdownlint (#608)

* Expand country brief and CII signal coverage (#611)

* perf(rss): raise news refresh interval to 10min and cache TTL to 20min (#612)

Reduces upstream RSS polling frequency and extends client-side cache
lifetime to lower API load and bandwidth usage.

* fix(api): harden cache-control headers for polymarket and rss-proxy (#613)

* docs(changelog): add v2.5.21 entry covering 86 merged PRs (#616)

Comprehensive changelog for 2026-03-01 release including Iran Attacks
layer, Telegram Intel panel, OREF sirens, GPS jamming, AviationStack,
breaking news alerts, and strategic risk score.

* fix(aviation): increase cache TTL from 30min to 2h to reduce API quota usage (#617)

* feat(oref): show history waves timeline with translation and NaN fix (#618)

- Fetch and display alert history waves in OrefSirensPanel (cap 50 most recent)
- Last-hour waves highlighted with amber border and RECENT badge
- Translate Hebrew history alerts via existing translateAlerts pipeline
- Guard formatAlertTime/formatWaveTime against NaN from unparseable OREF dates
- Cap relay history bootstrap to 500 records
- Add 3-minute TTL to prevent re-fetching history on every 10s poll
- Remove dead .oref-footer/.oref-history CSS; add i18n key for history summary

* feat(map): native mobile map experience with location detection and full feature parity (#619)

- Fix URL restore: lat/lon now override view center when explicitly provided
- Fix touch scroll: 8px threshold before drag activation, preventDefault once active
- Add location bootstrap: timezone-first detection, optional geolocation upgrade
- Enable DeckGL on mobile with deviceMemory capability guard
- Add DeckGL state sync on moveend/zoomend for URL param updates
- Fix breakpoint off-by-one: JS now uses <= to match CSS max-width: 768px
- Add country-click on SVG fallback with CSS transform inversion
- Add fitCountry() to both map engines (DeckGL uses fitBounds, SVG uses projection)
- Add SVG inertial touch animation with exponential velocity decay
- Add mobile map e2e tests for timezone, URL restore, touch, and breakpoint

* fix(sentry): guard pauseVideo optional chaining + add 4 noise filters (#624)

- Fix: `this.player.pauseVideo()` → `this.player.pauseVideo?.()` at
  LiveNewsPanel line 1301 (6 Sentry issues, 33 events)
- Noise: Chrome extension "Could not establish connection"
- Noise: Safari "webkitCurrentPlaybackTargetIsWireless" internal
- Noise: Sentry SDK crash on iOS "a.theme"
- Noise: Broaden Can't find variable to include EmptyRanges
- Noise: Catch stale cached builds with /this.player.\w+ is not a function/

* fix(aviation): prevent AviationStack API quota blowout (#623)

- Replace manual getCachedJson/setCachedJson with cachedFetchJson for
  intl delays — prevents thundering herd (concurrent misses each firing
  93 API calls independently)
- Bump Redis cache TTL from 30min to 2h
- Bump frontend polling from 10min to 2h to match server cache
- Bump circuit breaker browser cache from 5min to 2h
- Bump CDN edge tier from medium (5min) to static (1h)
- Bump cache key to v3 to force fresh entry with new TTL

* feat(news): server-side feed aggregation to reduce edge invocations by ~95% (#622)

Phase 1: Force CDN caching on rss-proxy (s-maxage=300 for 2xx, short
TTL for errors) — fixes bug where upstream no-cache headers were passed
through verbatim, defeating Vercel CDN.

Phase 2: Add ListFeedDigest RPC that aggregates all feeds server-side
into a single Redis-cached response. Client makes 1 request instead of
~90 per cycle. Includes circuit breaker with persistent cache fallback,
per-feed AI reclassification, and headline ingestion parity.

Phase 3: Increase polling interval from 5min to 7min to offset CDN
cache alignment.

New files:
- proto/worldmonitor/news/v1/list_feed_digest.proto
- server/worldmonitor/news/v1/{_feeds,_classifier,list-feed-digest}.ts
- src/services/ai-classify-queue.ts (extracted from rss.ts)

* feat(rss): add conditional GET (ETag/If-Modified-Since) to Railway relay (#625)

When RSS cache expires, send If-None-Match/If-Modified-Since headers
on revalidation. Upstream 304 responses refresh the cache timestamp
and serve cached body with zero bandwidth, cutting egress ~80-95%
for feeds that support conditional GET.

* fix(map): sync layer toggles to URL for shareable links (#576) (#621)

Layer toggles were not updating the browser URL, so shared links
would not carry the user's current layer state. Extracted the
debounced URL sync to a reusable class method and call it on
every layer change.

* chore(cache): bump all sub-5min cache TTLs and polling intervals (#626)

Audit and raise every cache/poll under 5 min to reduce upstream API
pressure and Vercel function invocations across 31 files:

Frontend polling: markets 4→8min, crypto 4→8min, predictions 5→10min,
natural 5→1hr, cableHealth 5→2hr, oref 10s→2min, maritime 30s→5min,
IntelligenceGapBadge 1→3min.

Circuit breakers: PizzINT 2→30min, aviation 5→20min, seismology/weather/
outages/statuses/wildfires 5→30min, military-vessels/flights/GDACS/
maritime/polymarket/GDELT 5→10min, chokepoint 5→20min.

Server Redis: market-quotes 2→8min, stablecoins 2→8min/5→10min,
vessel-snapshot 10s→5min, earthquakes 5→30min, sector 5→10min,
predictions/crypto/commodities 5→10min, outages/statuses 5→30min,
macro-signals/chokepoints 5→15min, aviation-sim 5→15min.

CDN edge: market RPCs fast→medium, infra/seismology fast→slow.

* fix(military): narrow ICAO hex ranges to stop civilian false positives (#627)

* fix(military): narrow hex ranges and callsign regex to stop civilian false positives (#462)

MILITARY_HEX_RANGES used entire country ICAO allocations instead of
military-specific sub-ranges (sourced from tar1090-db/ranges.json).
This flagged ALL commercial aircraft from Italy, Spain, Japan, India,
South Korea, etc. as military activity.

Key changes:
- Remove A00000-A3FFFF (US civilian N-numbers) — military starts at ADF7C8
- Italy 300000-33FFFF → 33FF00-33FFFF (top 256 codes only)
- Spain 340000-37FFFF → 350000-37FFFF (upper 3/4 confirmed military)
- Japan 840000-87FFFF removed (no confirmed JASDF sub-range)
- France narrowed to 3AA000-3AFFFF + 3B7000-3BFFFF
- Germany narrowed to 3EA000-3EBFFF + 3F4000-3FBFFF
- India 800000-83FFFF → 800200-8002FF (256 codes)
- Canada C00000-C0FFFF → C20000-C3FFFF (upper half)
- Remove unconfirmed: South Korea, Sweden, Singapore, Pakistan
- Add confirmed: Austria, Belgium, Switzerland, Brazil
- Drop overly broad /^[A-Z]{4,}\d{1,3}$/ callsign regex from server

* fix(military): constrain short prefixes + add classification tests

Move ambiguous 2-letter prefixes (AE, RF, TF, PAT, SAM, OPS, CTF,
IRG, TAF) to SHORT_MILITARY_PREFIXES — these now only match when
followed by a digit (e.g. AE1234=military, AEE123=Aegean Airlines).

Add 97-case test suite covering:
- Military callsign detection (19 known patterns)
- Short prefix digit gating (6 cases)
- Civilian airline non-detection (26 airlines)
- Short prefix letter rejection (6 cases)
- Military hex range boundaries (7 confirmed ranges)
- Civilian hex non-detection (19 codes)
- Boundary precision (ADF7C8 start, 33FF00 start, etc.)
- No-full-allocation guard (10 countries)

* fix: use charAt() instead of bracket indexing for strict TS

* fix(sidecar): add AVIATIONSTACK_API and ICAO_API_KEY to env allowlist (#632)

Both keys were added to Rust SUPPORTED_SECRET_KEYS and runtime-config.ts
but the sidecar's own ALLOWED_ENV_KEYS was never updated. This caused
"key not in allowlist" 403 when saving/verifying these keys from the
desktop settings UI.

Also adds AviationStack API validation in validateSecretAgainstProvider.

* fix(linux): sanitize env for xdg-open in AppImage (#631)

* fix(desktop): backoff on errors to stop CPU abuse + shrink settings window (#633)

Three bugs combine to burn 130% CPU when sidecar auth fails:

1. RefreshScheduler resets backoff multiplier to 1 (fastest) on error,
   causing failed endpoints to poll at base interval instead of backing off.
   Fix: exponential backoff on errors, same as unchanged-data path.

2. classify-event batch system ignores 401 (auth failure) — only pauses
   on 429/5xx. Hundreds of classify calls fire every 2s, each wasted.
   Fix: pause 120s on 401, matching the 429/5xx pattern.

3. Fetch patch retries every 401 (refresh token + retry), doubling all
   requests to the sidecar even when token refresh consistently fails.
   Fix: 60s cooldown after a retry-401 still returns 401.

Also shrinks settings window from 760→600px (min 620→480) to reduce
the empty whitespace below content on all tabs.

* feat: implement deduct situation feature (#636)

* feat: Add new intelligence panels for situation deduction, strategic posture, and tech events with associated backend services and E2E tests.

* fix(deduction): desktop-only gating, XSS fix, and code review fixes

- Gate DeductionPanel creation behind isDesktopApp check
- Gate deduce buttons in StrategicPosturePanel and TechEventsPanel behind isDesktopRuntime()
- Remove deduction from FULL_PANELS (runtime-only on desktop)
- Add DOMPurify to sanitize marked.parse() output before innerHTML
- Revert shared _shared.ts: restore UPSTREAM_TIMEOUT_MS=30_000, hardcoded GROQ_API_URL/MODEL
- Define local DEDUCT_TIMEOUT_MS=120_000, DEDUCT_API_URL, DEDUCT_MODEL in handler
- Fix dead API key guard: remove 'ollama-local' fallback, skip if no key
- Strip all console.log from deduct-situation handler (keep console.error)
- Bump max_tokens 500→1500 for meaningful analysis
- Fix cache key: deduct:sebuf→deduct:situation
- Add 5s client-side rate limit cooldown on submit button
- Move @types/marked to devDependencies, add dompurify + @types/dompurify
- Add DeductContextDetail shared interface in types/index.ts
- Extract buildNewsContext helper to src/utils/news-context.ts
- Pass getLatestNews to StrategicPosturePanel constructor
- Fix E2E test: .deduction-markdown-content→.deduction-result
- Revert emoji escapes in commands.ts to unicode format
- Revert variant-switcher whitespace in panel-layout.ts

* fix: resolve IranEvent timestamp type mismatches (string vs number)

Generated IranEvent.timestamp is string but consumers expected number.
- Coerce timestamps via Number() in data-loader before passing to aggregators
- Accept string|number in MapPopup IranEventPopupData interface
- Fix scrapedAt fallback type in conflict/index.ts
- Coerce timestamp in country-intel.ts and DeckGLMap.ts sort

* fix: restore generated code from main, keep intelligence deductSituation RPC

Reverts all src/generated/ files to origin/main except the intelligence
service client/server which retains the new deductSituation RPC added by
this PR. Fixes scrapedAt type to match generated number type.

* fix: add missing listGulfQuotes handler from main

The Gulf Economies panel (PR #667) added this RPC after the feature
branch was forked. Bring handler and registration from main.

* fix: address code review findings (P1+P2+P3)

- P1: Fix event listener leak — store handler ref, remove in destroy()
- P1: Prevent cooldown bypass via requestSubmit when button disabled
- P2: Read process.env inside function body for runtime secret updates
- P3: Use \u{1F9E0} unicode escape in StrategicPosturePanel (consistency)

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
Co-authored-by: Sebastien Melki <sebastien@anghami.com>
Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Konstanty Szumigaj <kostekszumigaj@gmail.com>
Co-authored-by: Chris Chen <fuleinist@users.noreply.github.com>
Co-authored-by: facusturla <facusturla@users.noreply.github.com>
Co-authored-by: karim <mirakijka@gmail.com>
Co-authored-by: maxime.io <maxime.de.visscher@gmail.com>
Co-authored-by: Elias El Khoury <efk@anghami.com>
Co-authored-by: Fayez Bast <96446332+FayezBast@users.noreply.github.com>
 2026-03-01 20:49:55 +04:00
Elie Habib
36e36d8b57 Cost/traffic hardening, runtime fallback controls, and PostHog removal (#638) 
- Remove PostHog analytics runtime and configuration
- Add API rate limiting (api/_rate-limit.js)
- Harden traffic controls across edge functions
- Add runtime fallback controls and data-loader improvements
- Add military base data scripts (fetch-mirta-bases, fetch-osm-bases)
- Gitignore large raw data files
- Settings playground prototypes
 2026-03-01 11:53:20 +04:00
Elie Habib
cac2a4f5af fix(desktop): route register-interest to cloud when sidecar lacks CONVEX_URL (#639) 
* fix(desktop): route register-interest to cloud when sidecar lacks CONVEX_URL

The waitlist registration endpoint needs Convex (cloud-only dependency).
The sidecar handler returned 503 without cloud fallback, and
getRemoteApiBaseUrl() returned '' on desktop (VITE_WS_API_URL unset),
so the settings window fetch resolved to tauri://localhost → 404.

Three-layer fix:
1. Sidecar: tryCloudFallback() when CONVEX_URL missing (proxies to
   https://worldmonitor.app via remoteBase)
2. runtime.ts: getRemoteApiBaseUrl() defaults to https://worldmonitor.app
   on desktop when VITE_WS_API_URL is unset
3. CI: add VITE_WS_API_URL=https://worldmonitor.app to all 4 desktop
   build steps

* chore(deps): bump posthog-js to fix pre-push typecheck
 2026-03-01 11:46:31 +04:00
Elie Habib
5bb3696f7a chore: bump version to 2.5.21 (#605) 2026-03-01 02:40:41 +04:00