eliott/worldmonitor
1
0
Fork 0
mirror of https://github.com/koala73/worldmonitor.git synced 2026-04-25 17:14:57 +02:00
Code Issues Packages Projects Releases Wiki Activity
3,613 Commits 138 Branches 48 Tags
main
Commit Graph

6 Commits

Author SHA1 Message Date
Elie Habib
5c955691a9 feat(energy-atlas): live tanker map layer + contract (parity PR 3, plan U7-U8) (#3402) 
* feat(energy-atlas): live tanker map layer + contract (PR 3, plan U7-U8)

Lands the third and final parity-push surface — per-vessel tanker positions
inside chokepoint bounding boxes, refreshed every 60s. Closes the visual
gap with peer reference energy-intel sites for the live AIS tanker view.

Per docs/plans/2026-04-25-003-feat-energy-parity-pushup-plan.md PR 3.
Codex-approved through 8 review rounds against origin/main @ 050073354.

U7 — Contract changes (relay + handler + proto + gateway + rate-limit + test):

- scripts/ais-relay.cjs: parallel `tankerReports` Map populated for AIS
  ship type 80-89 (tanker class) per ITU-R M.1371. SEPARATE from the
  existing `candidateReports` Map (military-only) so the existing
  military-detection consumer's contract stays unchanged. Snapshot
  endpoint extended to accept `bbox=swLat,swLon,neLat,neLon` + `tankers=true`
  query params, with bbox-filtering applied server-side. Tanker reports
  cleaned up on the same retention window as candidate reports; capped
  at 200 per response (10× headroom for global storage).
- proto/worldmonitor/maritime/v1/{get_,}vessel_snapshot.proto:
  - new `bool include_tankers = 6` request field
  - new `repeated SnapshotCandidateReport tanker_reports = 7` response
    field (reuses existing message shape; parallel to candidate_reports)
- server/worldmonitor/maritime/v1/get-vessel-snapshot.ts: REPLACES the
  prior 5-minute `with|without` cache with a request-keyed cache —
  (includeCandidates, includeTankers, quantizedBbox) — at 60s TTL for
  the live-tanker path and 5min TTL for the existing density/disruption
  consumers. Also adds 1° bbox quantization for cache-key reuse and a
  10° max-bbox guard (BboxTooLargeError) to prevent malicious clients
  from pulling all tankers through one query.
- server/gateway.ts: NEW `'live'` cache tier. CacheTier union extended;
  TIER_HEADERS + TIER_CDN_CACHE both gain entries with `s-maxage=60,
  stale-while-revalidate=60`. RPC_CACHE_TIER maps the maritime endpoint
  from `'no-store'` to `'live'` so the CDN absorbs concurrent identical
  requests across all viewers (without this, N viewers × 6 chokepoints
  hit AISStream upstream linearly).
- server/_shared/rate-limit.ts: ENDPOINT_RATE_POLICIES entry for the
  maritime endpoint at 60 req/min/IP — enough headroom for one user's
  6-chokepoint tab plus refreshes; flags only true scrape-class traffic.
- tests/route-cache-tier.test.mjs: regex extended to include `live` so
  the every-route-has-an-explicit-tier check still recognises the new
  mapping. Without this, the new tier would silently drop the maritime
  route from the validator's route map.

U8 — LiveTankersLayer consumer:

- src/services/live-tankers.ts: per-chokepoint fetcher with 60s in-memory
  cache. Promise.allSettled — never .all — so one chokepoint failing
  doesn't blank the whole layer (failed zones serve last-known data).
  Sources bbox centroids from src/config/chokepoint-registry.ts
  (CORRECT location — server/.../​_chokepoint-ids.ts strips lat/lon).
  Default chokepoint set: hormuz_strait, suez, bab_el_mandeb,
  malacca_strait, panama, bosphorus.
- src/components/DeckGLMap.ts: new `createLiveTankersLayer()` ScatterplotLayer
  styled by speed (anchored amber when speed < 0.5 kn, underway cyan,
  unknown gray); new `loadLiveTankers()` async loader with abort-controller
  cancellation. Layer instantiated when `mapLayers.liveTankers && this.liveTankers.length > 0`.
- src/config/map-layer-definitions.ts: `LayerDefinition` for `liveTankers`
  with `renderers: ['flat'], deckGLOnly: true` (matches existing
  storageFacilities/fuelShortages pattern). Added to `VARIANT_LAYER_ORDER.energy`
  near `ais` so getLayersForVariant() and sanitizeLayersForVariant()
  include it on the energy variant — without this addition the layer
  would be silently stripped even when toggled on.
- src/types/index.ts: `liveTankers?: boolean` on the MapLayers union.
- src/config/panels.ts: ENERGY_MAP_LAYERS + ENERGY_MOBILE_MAP_LAYERS
  both gain `liveTankers: true`. Default `false` everywhere else.
- src/services/maritime/index.ts: existing snapshot consumer pinned to
  `includeTankers: false` to satisfy the proto's new required field;
  preserves identical behavior for the AIS-density / military-detection
  surfaces.

Tests:
- npm run typecheck clean.
- 5 unit tests in tests/live-tankers-service.test.mjs cover the default
  chokepoint set (rejects ids that aren't in CHOKEPOINT_REGISTRY), the
  60s cache TTL pin (must match gateway 'live' tier s-maxage), and bbox
  derivation (±2° padding, total span under the 10° handler guard).
- tests/route-cache-tier.test.mjs continues to pass after the regex
  extension; the new maritime tier is correctly extracted.

Defense in depth:
- THREE-layer cache (CDN 'live' tier → handler bbox-keyed 60s → service
  in-memory 60s) means concurrent users hit the relay sub-linearly.
- Server-side 200-vessel cap on tanker_reports + client-side cap;
  protects layer render perf even on a runaway relay payload.
- Bbox-size guard (10° max) prevents a single global-bbox query from
  exfiltrating every tanker.
- Per-IP rate limit at 60/min covers normal use; flags scrape-class only.
- Existing military-detection contract preserved: `candidate_reports`
  field semantics unchanged; consumers self-select via include_tankers
  vs include_candidates rather than the response field changing meaning.

* fix(energy-atlas): wire LiveTankers loop + 400 bbox-range guard (PR3 review)

Three findings from review of #3402:

P1 — loadLiveTankers() was never called (DeckGLMap.ts:2999):
- Add ensureLiveTankersLoop() / stopLiveTankersLoop() helpers paired with
  the layer-enabled / layer-disabled branches in updateLayers(). The
  ensure helper kicks an immediate load + a 60s setInterval; idempotent
  so calling it on every layers update is safe.
- Wire stopLiveTankersLoop() into destroy() and into the layer-disabled
  branch so we don't hammer the relay when the layer is off.
- Layer factory now runs only when liveTankers.length > 0; ensureLoop
  fires on every observed-enabled tick so first-paint kicks the load
  even before the first tanker arrives.

P1 — bbox lat/lon range guard (get-vessel-snapshot.ts:253):
- Out-of-range bboxes (e.g. ne_lat=200) previously passed the size
  guard (200-195=5° < 10°) but failed at the relay, which silently
  drops the bbox param and returns a global capped subset — making
  the layer appear to "work" with stale phantom data.
- Add isValidLatLon() check inside extractAndValidateBbox(): every
  corner must satisfy [-90, 90] / [-180, 180] before the size guard
  runs. Failure throws BboxValidationError.

P2 — BboxTooLargeError surfaced as 500 instead of 400:
- server/error-mapper.ts maps errors to HTTP status by checking
  `'statusCode' in error`. The previous BboxTooLargeError extended
  Error without that property, so the mapper fell through to
  "unhandled error" → 500.
- Rename to BboxValidationError, add `readonly statusCode = 400`.
  Mapper now surfaces it as HTTP 400 with a descriptive reason.
- Keep BboxTooLargeError as a backwards-compat alias so existing
  imports / tests don't break.

Tests:
- Updated tests/server-handlers.test.mjs structural test to pin the
  new class name + statusCode + lat/lon range checks. 24 tests pass.
- typecheck (src + api) clean.

* fix(energy-atlas): thread AbortSignal through fetchLiveTankers (PR3 review #2)

P2 — AbortController was created + aborted but signal was never passed
into the actual fetch path (DeckGLMap.ts:3048 / live-tankers.ts:100):
- Toggling the layer off, destroying the map, or starting a new refresh
  did not actually cancel in-flight network work. A slow older refresh
  could complete after a newer one and overwrite this.liveTankers with
  stale data.

Threading:
- fetchLiveTankers() now accepts `options.signal: AbortSignal`. Signal
  is passed through to client.getVesselSnapshot() per chokepoint via
  the Connect-RPC client's standard `{ signal }` option.
- Per-zone abort handling: bail early if signal is already aborted
  before the fetch starts (saves a wasted RPC + cache write); re-check
  after the fetch resolves so a slow resolver can't clobber cache
  after the caller cancelled.

Stale-result race guard in DeckGLMap.loadLiveTankers:
- Capture controller in a local before storing on this.liveTankersAbort.
- After fetchLiveTankers resolves, drop the result if EITHER:
  - controller.signal is now aborted (newer load cancelled this one)
  - this.liveTankersAbort points to a different controller (a newer
    load already started + replaced us in the field)
- Without these guards, an older fetch that completed despite
  signal.aborted could still write to this.liveTankers and call
  updateLayers, racing with the newer load.

Tests: 1 new signature-pin test in tests/live-tankers-service.test.mts
verifies fetchLiveTankers accepts options.signal — guards against future
edits silently dropping the parameter and re-introducing the race.
6 tests pass. typecheck clean.

* fix(energy-atlas): bound vessel-snapshot cache via LRU eviction (PR3 review)

Greptile P2 finding: the in-process cache Map grows unbounded across the
serverless instance lifetime. Each distinct (includeCandidates,
includeTankers, quantizedBbox) triple creates a slot that's never evicted.
With 1° quantization and a misbehaving client the keyspace is ~64,000
entries — realistic load is ~12, so a 128-slot cap leaves 10x headroom
while making OOM impossible.

Implementation:
- SNAPSHOT_CACHE_MAX_SLOTS = 128.
- evictIfNeeded() walks insertion order and evicts the first slot whose
  inFlight is null. Slots with active fetches are skipped to avoid
  orphaning awaiting callers; we accept brief over-cap growth until
  in-flight settles.
- touchSlot() re-inserts a slot at the end of Map insertion order on
  hit / in-flight join / fresh write so it counts as most-recently-used.
 2026-04-25 17:56:23 +04:00
Sebastien Melki
58e42aadf9 chore(api): enforce sebuf contract + migrate drifting endpoints (#3207) (#3242) 
* chore(api): enforce sebuf contract via exceptions manifest (#3207)

Adds api/api-route-exceptions.json as the single source of truth for
non-proto /api/ endpoints, with scripts/enforce-sebuf-api-contract.mjs
gating every PR via npm run lint:api-contract. Fixes the root-only blind
spot in the prior allowlist (tests/edge-functions.test.mjs), which only
scanned top-level *.js files and missed nested paths and .ts endpoints —
the gap that let api/supply-chain/v1/country-products.ts and friends
drift under proto domain URL prefixes unchallenged.

Checks both directions: every api/<domain>/v<N>/[rpc].ts must pair with
a generated service_server.ts (so a deleted proto fails CI), and every
generated service must have an HTTP gateway (no orphaned generated code).

Manifest entries require category + reason + owner, with removal_issue
mandatory for temporary categories (deferred, migration-pending) and
forbidden for permanent ones. .github/CODEOWNERS pins the manifest to
@SebastienMelki so new exceptions don't slip through review.

The manifest only shrinks: migration-pending entries (19 today) will be
removed as subsequent commits in this PR land each migration.

* refactor(maritime): migrate /api/ais-snapshot → maritime/v1.GetVesselSnapshot (#3207)

The proto VesselSnapshot was carrying density + disruptions but the frontend
also needed sequence, relay status, and candidate_reports to drive the
position-callback system. Those only lived on the raw relay passthrough, so
the client had to keep hitting /api/ais-snapshot whenever callbacks were
registered and fall back to the proto RPC only when the relay URL was gone.

This commit pushes all three missing fields through the proto contract and
collapses the dual-fetch-path into one proto client call.

Proto changes (proto/worldmonitor/maritime/v1/):
  - VesselSnapshot gains sequence, status, candidate_reports.
  - GetVesselSnapshotRequest gains include_candidates (query: include_candidates).

Handler (server/worldmonitor/maritime/v1/get-vessel-snapshot.ts):
  - Forwards include_candidates to ?candidates=... on the relay.
  - Separate 5-min in-memory caches for the candidates=on and candidates=off
    variants; they have very different payload sizes and should not share a slot.
  - Per-request in-flight dedup preserved per-variant.

Frontend (src/services/maritime/index.ts):
  - fetchSnapshotPayload now calls MaritimeServiceClient.getVesselSnapshot
    directly with includeCandidates threaded through. The raw-relay path,
    SNAPSHOT_PROXY_URL, DIRECT_RAILWAY_SNAPSHOT_URL and LOCAL_SNAPSHOT_FALLBACK
    are gone — production already routed via Vercel, the "direct" branch only
    ever fired on localhost, and the proto gateway covers both.
  - New toLegacyCandidateReport helper mirrors toDensityZone/toDisruptionEvent.

api/ais-snapshot.js deleted; manifest entry removed. Only reduced the codegen
scope to worldmonitor.maritime.v1 (buf generate --path) — regenerating the
full tree drops // @ts-nocheck from every client/server file and surfaces
pre-existing type errors across 30+ unrelated services, which is not in
scope for this PR.

Shape-diff vs legacy payload:
  - disruptions / density: proto carries the same fields, just with the
    GeoCoordinates wrapper and enum strings (remapped client-side via
    existing toDisruptionEvent / toDensityZone helpers).
  - sequence, status.{connected,vessels,messages}: now populated from the
    proto response — was hardcoded to 0/false in the prior proto fallback.
  - candidateReports: same shape; optional numeric fields come through as
    0 instead of undefined, which the legacy consumer already handled.

* refactor(sanctions): migrate /api/sanctions-entity-search → LookupSanctionEntity (#3207)

The proto docstring already claimed "OFAC + OpenSanctions" coverage but the
handler only fuzzy-matched a local OFAC Redis index — narrower than the
legacy /api/sanctions-entity-search, which proxied OpenSanctions live (the
source advertised in docs/api-proxies.mdx). Deleting the legacy without
expanding the handler would have been a silent coverage regression for
external consumers.

Handler changes (server/worldmonitor/sanctions/v1/lookup-entity.ts):
  - Primary path: live search against api.opensanctions.org/search/default
    with an 8s timeout and the same User-Agent the legacy edge fn used.
  - Fallback path: the existing OFAC local fuzzy match, kept intact for when
    OpenSanctions is unreachable / rate-limiting.
  - Response source field flips between 'opensanctions' (happy path) and
    'ofac' (fallback) so clients can tell which index answered.
  - Query validation tightened: rejects q > 200 chars (matches legacy cap).

Rate limiting:
  - Added /api/sanctions/v1/lookup-entity to ENDPOINT_RATE_POLICIES at 30/min
    per IP — matches the legacy createIpRateLimiter budget. The gateway
    already enforces per-endpoint policies via checkEndpointRateLimit.

Docs:
  - docs/api-proxies.mdx — dropped the /api/sanctions-entity-search row
    (plus the orphaned /api/ais-snapshot row left over from the previous
    commit in this PR).
  - docs/panels/sanctions-pressure.mdx — points at the new RPC URL and
    describes the OpenSanctions-primary / OFAC-fallback semantics.

api/sanctions-entity-search.js deleted; manifest entry removed.

* refactor(military): migrate /api/military-flights → ListMilitaryFlights (#3207)

Legacy /api/military-flights read a pre-baked Redis blob written by the
seed-military-flights cron and returned flights in a flat app-friendly
shape (lat/lon, lowercase enums, lastSeenMs). The proto RPC takes a bbox,
fetches OpenSky live, classifies server-side, and returns nested
GeoCoordinates + MILITARY_*_TYPE_* enum strings + lastSeenAt — same data,
different contract.

fetchFromRedis in src/services/military-flights.ts was doing nothing
sebuf-aware. Renamed it to fetchViaProto and rewrote to:

  - Instantiate MilitaryServiceClient against getRpcBaseUrl().
  - Iterate MILITARY_QUERY_REGIONS (PACIFIC + WESTERN) in parallel — same
    regions the desktop OpenSky path and the seed cron already use, so
    dashboard coverage tracks the analytic pipeline.
  - Dedup by hexCode across regions.
  - Map proto → app shape via new mapProtoFlight helper plus three reverse
    enum maps (AIRCRAFT_TYPE_REVERSE, OPERATOR_REVERSE, CONFIDENCE_REVERSE).

The seed cron (scripts/seed-military-flights.mjs) stays put: it feeds
regional-snapshot mobility, cross-source signals, correlation, and the
health freshness check (api/health.js: 'military:flights:v1'). None of
those read the legacy HTTP endpoint; they read the Redis key directly.
The proto handler uses its own per-bbox cache keys under the same prefix,
so dashboard traffic no longer races the seed cron's blob — the two paths
diverge by a small refresh lag, which is acceptable.

Docs: dropped the /api/military-flights row from docs/api-proxies.mdx.

api/military-flights.js deleted; manifest entry removed.

Shape-diff vs legacy:
  - f.location.{latitude,longitude} → f.lat, f.lon
  - f.aircraftType: MILITARY_AIRCRAFT_TYPE_TANKER → 'tanker' via reverse map
  - f.operator: MILITARY_OPERATOR_USAF → 'usaf' via reverse map
  - f.confidence: MILITARY_CONFIDENCE_LOW → 'low' via reverse map
  - f.lastSeenAt (number) → f.lastSeen (Date)
  - f.enrichment → f.enriched (with field renames)
  - Extra fields registration / aircraftModel / origin / destination /
    firstSeenAt now flow through where proto populates them.

* fix(supply-chain): thread includeCandidates through chokepoint status (#3207)

Caught by tsconfig.api.json typecheck in the pre-push hook (not covered
by the plain tsc --noEmit run that ran before I pushed the ais-snapshot
commit). The chokepoint status handler calls getVesselSnapshot internally
with a static no-auth request — now required to include the new
includeCandidates bool from the proto extension.

Passing false: server-internal callers don't need per-vessel reports.

* test(maritime): update getVesselSnapshot cache assertions (#3207)

The ais-snapshot migration replaced the single cachedSnapshot/cacheTimestamp
pair with a per-variant cache so candidates-on and candidates-off payloads
don't evict each other. Pre-push hook surfaced that tests/server-handlers
still asserted the old variable names. Rewriting the assertions to match
the new shape while preserving the invariants they actually guard:

  - Freshness check against slot TTL.
  - Cache read before relay call.
  - Per-slot in-flight dedup.
  - Stale-serve on relay failure (result ?? slot.snapshot).

* chore(proto): restore // @ts-nocheck on regenerated maritime files (#3207)

I ran 'buf generate --path worldmonitor/maritime/v1' to scope the proto
regen to the one service I was changing (to avoid the toolchain drift
that drops @ts-nocheck from 60+ unrelated files — separate issue). But
the repo convention is the 'make generate' target, which runs buf and
then sed-prepends '// @ts-nocheck' to every generated .ts file. My
scoped command skipped the sed step. The proto-check CI enforces the
sed output, so the two maritime files need the directive restored.

* refactor(enrichment): decomm /api/enrichment/{company,signals} legacy edge fns (#3207)

Both endpoints were already ported to IntelligenceService:
  - getCompanyEnrichment  (/api/intelligence/v1/get-company-enrichment)
  - listCompanySignals    (/api/intelligence/v1/list-company-signals)

No frontend callers of the legacy /api/enrichment/* paths exist. Removes:
  - api/enrichment/company.js, signals.js, _domain.js
  - api-route-exceptions.json migration-pending entries (58 remain)
  - docs/api-proxies.mdx rows for /api/enrichment/{company,signals}
  - docs/architecture.mdx reference updated to the IntelligenceService RPCs

Verified: typecheck, typecheck:api, lint:api-contract (89 files / 58 entries),
lint:boundaries, tests/edge-functions.test.mjs (136 pass),
tests/enrichment-caching.test.mjs (14 pass — still guards the intelligence/v1
handlers), make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(leads): migrate /api/{contact,register-interest} → LeadsService (#3207)

New leads/v1 sebuf service with two POST RPCs:
  - SubmitContact    → /api/leads/v1/submit-contact
  - RegisterInterest → /api/leads/v1/register-interest

Handler logic ported 1:1 from api/contact.js + api/register-interest.js:
  - Turnstile verification (desktop sources bypass, preserved)
  - Honeypot (website field) silently accepts without upstream calls
  - Free-email-domain gate on SubmitContact (422 ApiError)
  - validateEmail (disposable/offensive/typo-TLD/MX) on RegisterInterest
  - Convex writes via ConvexHttpClient (contactMessages:submit, registerInterest:register)
  - Resend notification + confirmation emails (HTML templates unchanged)

Shared helpers moved to server/_shared/:
  - turnstile.ts (getClientIp + verifyTurnstile)
  - email-validation.ts (disposable/offensive/MX checks)

Rate limits preserved via ENDPOINT_RATE_POLICIES:
  - submit-contact:    3/hour per IP (was in-memory 3/hr)
  - register-interest: 5/hour per IP (was in-memory 5/hr; desktop
    sources previously capped at 2/hr via shared in-memory map —
    now 5/hr like everyone else, accepting the small regression in
    exchange for Upstash-backed global limiting)

Callers updated:
  - pro-test/src/App.tsx contact form → new submit-contact path
  - src-tauri/sidecar/local-api-server.mjs cloud-fallback rewrites
    /api/register-interest → /api/leads/v1/register-interest when
    proxying; keeps local path for older desktop builds
  - src/services/runtime.ts isKeyFreeApiTarget allows both old and
    new paths through the WORLDMONITOR_API_KEY-optional gate

Tests:
  - tests/contact-handler.test.mjs rewritten to call submitContact
    handler directly; asserts on ValidationError / ApiError
  - tests/email-validation.test.mjs + tests/turnstile.test.mjs
    point at the new server/_shared/ modules

Deleted: api/contact.js, api/register-interest.js, api/_ip-rate-limit.js,
api/_turnstile.js, api/_email-validation.js, api/_turnstile.test.mjs.
Manifest entries removed (58 → 56). Docs updated (api-platform,
api-commerce, usage-rate-limits).

Verified: npm run typecheck + typecheck:api + lint:api-contract
(88 files / 56 entries) + lint:boundaries pass; full test:data
(5852 tests) passes; make generate is zero-diff.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore(pro-test): rebuild bundle for leads/v1 contact form (#3207)

Updates the enterprise contact form to POST to /api/leads/v1/submit-contact
(old path /api/contact removed in the previous commit).

Bundle is rebuilt from pro-test/src/App.tsx source change in 9ccd309d.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): address HIGH review findings 1-3 (#3207)

Three review findings from @koala73 on the sebuf-migration PR, all
silent bugs that would have shipped to prod:

### 1. Sanctions rate-limit policy was dead code

ENDPOINT_RATE_POLICIES keyed the 30/min budget under
/api/sanctions/v1/lookup-entity, but the generated route (from the
proto RPC LookupSanctionEntity) is /api/sanctions/v1/lookup-sanction-entity.
hasEndpointRatePolicy / getEndpointRatelimit are exact-string pathname
lookups, so the mismatch meant the endpoint fell through to the
generic 600/min global limiter instead of the advertised 30/min.

Net effect: the live OpenSanctions proxy endpoint (unauthenticated,
external upstream) had 20x the intended rate budget. Fixed by renaming
the policy key to match the generated route.

### 2. Lost stale-seed fallback on military-flights

Legacy api/military-flights.js cascaded military:flights:v1 →
military:flights:stale:v1 before returning empty. The new proto
handler went straight to live OpenSky/relay and returned null on miss.

Relay or OpenSky hiccup used to serve stale seeded data (24h TTL);
under the new handler it showed an empty map. Both keys are still
written by scripts/seed-military-flights.mjs on every run — fix just
reads the stale key when the live fetch returns null, converts the
seed's app-shape flights (flat lat/lon, lowercase enums, lastSeenMs)
to the proto shape (nested GeoCoordinates, enum strings, lastSeenAt),
and filters to the request bbox.

Read via getRawJson (unprefixed) to match the seed cron's writes,
which bypass the env-prefix system.

### 3. Hex-code casing mismatch broke getFlightByHex

The seed cron writes hexCode: icao24.toUpperCase() (uppercase);
src/services/military-flights.ts:getFlightByHex uppercases the lookup
input: f.hexCode === hexCode.toUpperCase(). The new proto handler
preserved OpenSky's lowercase icao24, and mapProtoFlight is a
pass-through. getFlightByHex was silently returning undefined for
every call after the migration.

Fix: uppercase in the proto handler (live + stale paths), and document
the invariant in a comment on MilitaryFlight.hex_code in
military_flight.proto so future handlers don't re-break it.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries) / lint:boundaries clean
- tests/edge-functions.test.mjs 130 pass
- make generate zero-diff (openapi spec regenerated for proto comment)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): restore desktop 2/hr rate cap on register-interest (#3207)

Addresses HIGH review finding #4 from @koala73. The legacy
api/register-interest.js applied a nested 2/hr per-IP cap when
`source === 'desktop-settings'`, on top of the generic 5/hr endpoint
budget. The sebuf migration lost this — desktop-source requests now
enjoy the full 5/hr cap.

Since `source` is an unsigned client-supplied field, anyone sending
`source: 'desktop-settings'` skips Turnstile AND gets 5/hr. Without
the tighter cap the Turnstile bypass is cheaper to abuse.

Added `checkScopedRateLimit` to `server/_shared/rate-limit.ts` — a
reusable second-stage Upstash limiter keyed on an opaque scope string
+ caller identifier. Fail-open on Redis errors to match existing
checkRateLimit / checkEndpointRateLimit semantics. Handlers that need
per-subscope caps on top of the gateway-level endpoint budget use this
helper.

In register-interest: when `isDesktopSource`, call checkScopedRateLimit
with scope `/api/leads/v1/register-interest#desktop`, limit=2, window=1h,
IP as identifier. On exceeded → throw ApiError(429).

### What this does not fix

This caps the blast radius of the Turnstile bypass but does not close
it — an attacker sending `source: 'desktop-settings'` still skips
Turnstile (just at 2/hr instead of 5/hr). The proper fix is a signed
desktop-secret header that authenticates the bypass; filed as
follow-up #3252. That requires coordinated Tauri build + Vercel env
changes out of scope for #3207.

### Verified

- typecheck + typecheck:api clean
- lint:api-contract (56 entries)
- tests/edge-functions.test.mjs + contact-handler.test.mjs (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review): MEDIUM + LOW + rate-limit-policy CI check (#3207)

Closes out the remaining @koala73 review findings from #3242 that
didn't already land in the HIGH-fix commits, plus the requested CI
check that would have caught HIGH #1 (dead-code policy key) at
review time.

### MEDIUM #5 — Turnstile missing-secret policy default

Flip `verifyTurnstile`'s default `missingSecretPolicy` from `'allow'`
to `'allow-in-development'`. Dev with no secret = pass (expected
local); prod with no secret = reject + log. submit-contact was
already explicitly overriding to `'allow-in-development'`;
register-interest was silently getting `'allow'`. Safe default now
means a future missing-secret misconfiguration in prod gets caught
instead of silently letting bots through. Removed the now-redundant
override in submit-contact.

### MEDIUM #6 — Silent enum fallbacks in maritime client

`toDisruptionEvent` mapped `AIS_DISRUPTION_TYPE_UNSPECIFIED` / unknown
enum values → `gap_spike` / `low` silently. Refactored to return null
when either enum is unknown; caller filters nulls out of the array.
Handler doesn't produce UNSPECIFIED today, but the `gap_spike`
default would have mislabeled the first new enum value the proto
ever adds — dropping unknowns is safer than shipping wrong labels.

### LOW — Copy drift in register-interest email

Email template hardcoded `435+ Sources`; PR #3241 bumped marketing to
`500+`. Bumped in the rewritten file to stay consistent.

The `as any` on Convex mutation names carried over from legacy and
filed as follow-up #3253.

### Rate-limit-policy coverage lint

`scripts/enforce-rate-limit-policies.mjs` validates every key in
`ENDPOINT_RATE_POLICIES` resolves to a proto-generated gateway route
by cross-referencing `docs/api/*.openapi.yaml`. Fails with the
sanctions-entity-search incident referenced in the error message so
future drift has a paper trail.

Wired into package.json (`lint:rate-limit-policies`) and the pre-push
hook alongside `lint:boundaries`. Smoke-tested both directions —
clean repo passes (5 policies / 175 routes), seeded drift (the exact
HIGH #1 typo) fails with the advertised remedy text.

### Verified
- `lint:rate-limit-policies` ✓
- `typecheck` + `typecheck:api` ✓
- `lint:api-contract` ✓ (56 entries)
- `lint:boundaries` ✓
- edge-functions + contact-handler tests (147 pass)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 5): decomm /api/eia/* + migrate /api/satellites → IntelligenceService (#3207)

Both targets turned out to be decomm-not-migration cases. The original
plan called for two new services (economic/v1.GetEiaSeries +
natural/v1.ListSatellitePositions) but research found neither was
needed:

### /api/eia/[[...path]].js — pure decomm, zero consumers

The "catch-all" is a misnomer — only two paths actually worked,
/api/eia/health and /api/eia/petroleum, both Redis-only readers.
Zero frontend callers in src/. Zero server-side readers. Nothing
consumes the `energy:eia-petroleum:v1` key that seed-eia-petroleum.mjs
writes daily.

The EIA data the frontend actually uses goes through existing typed
RPCs in economic/v1: GetEnergyPrices, GetCrudeInventories,
GetNatGasStorage, GetEnergyCapacity. None of those touch /api/eia/*.

Building GetEiaSeries would have been dead code. Deleted the legacy
file + its test (tests/api-eia-petroleum.test.mjs — it only covered
the legacy endpoint, no behavior to preserve). Empty api/eia/ dir
removed.

**Note for review:** the Redis seed cron keeps running daily and
nothing consumes it. If that stays unused, seed-eia-petroleum.mjs
should be retired too (separate PR). Out of scope for sebuf-migration.

### /api/satellites.js — Learning #2 strikes again

IntelligenceService.ListSatellites already exists at
/api/intelligence/v1/list-satellites, reads the same Redis key
(intelligence:satellites:tle:v1), and supports an optional country
filter the legacy didn't have.

One frontend caller in src/services/satellites.ts needed to switch
from `fetch(toApiUrl('/api/satellites'))` to the typed
IntelligenceServiceClient.listSatellites. Shape diff was tiny —
legacy `noradId` became proto `id` (handler line 36 already picks
either), everything else identical. alt/velocity/inclination in the
proto are ignored by the caller since it propagates positions
client-side via satellite.js.

Kept the client-side cache + failure cooldown + 20s timeout (still
valid concerns at the caller level).

### Manifest + docs
- api-route-exceptions.json: 56 → 54 entries (both removed)
- docs/api-proxies.mdx: dropped the two rows from the Raw-data
  passthroughs table

### Verified
- typecheck + typecheck:api ✓
- lint:api-contract (54 entries) / lint:boundaries / lint:rate-limit-policies ✓
- tests/edge-functions.test.mjs 127 pass (down from 130 — 3 tests were
  for the deleted eia endpoint)
- make generate zero-diff (no proto changes)

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 6): migrate /api/supply-chain/v1/{country-products,multi-sector-cost-shock} → SupplyChainService (#3207)

Both endpoints were hand-rolled TS handlers sitting under a proto URL prefix —
the exact drift the manifest guardrail flagged. Promoted both to typed RPCs:

- GetCountryProducts → /api/supply-chain/v1/get-country-products
- GetMultiSectorCostShock → /api/supply-chain/v1/get-multi-sector-cost-shock

Handlers preserve the existing semantics: PRO-gate via isCallerPremium(ctx.request),
iso2 / chokepointId validation, raw bilateral-hs4 Redis read (skip env-prefix to
match seeder writes), CHOKEPOINT_STATUS_KEY for war-risk tier, and the math from
_multi-sector-shock.ts unchanged. Empty-data and non-PRO paths return the typed
empty payload (no 403 — the sebuf gateway pattern is empty-payload-on-deny).

Client wrapper switches from premiumFetch to client.getCountryProducts/
client.getMultiSectorCostShock. Legacy MultiSectorShock / MultiSectorShockResponse /
CountryProductsResponse names remain as type aliases of the generated proto types
so CountryBriefPanel + CountryDeepDivePanel callsites compile with zero churn.

Manifest 54 → 52. Rate-limit gateway routes 175 → 177.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(gateway): add cache-tier entries for new supply-chain RPCs (#3207)

Pre-push tests/route-cache-tier.test.mjs caught the missing entries.
Both PRO-gated, request-varying — match the existing supply-chain PRO cohort
(get-country-cost-shock, get-bypass-options, etc.) at slow-browser tier.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 7): migrate /api/scenario/v1/{run,status,templates} → ScenarioService (#3207)

Promote the three literal-filename scenario endpoints to a typed sebuf
service with three RPCs:

  POST /api/scenario/v1/run-scenario        (RunScenario)
  GET  /api/scenario/v1/get-scenario-status (GetScenarioStatus)
  GET  /api/scenario/v1/list-scenario-templates (ListScenarioTemplates)

Preserves all security invariants from the legacy handlers:
- 405 for wrong method (sebuf service-config method gate)
- scenarioId validation against SCENARIO_TEMPLATES registry
- iso2 regex ^[A-Z]{2}$
- JOB_ID_RE path-traversal guard on status
- Per-IP 10/min rate limit (moved to gateway ENDPOINT_RATE_POLICIES)
- Queue-depth backpressure (>100 → 429)
- PRO gating via isCallerPremium
- AbortSignal.timeout on every Redis pipeline (runRedisPipeline helper)

Wire-level diffs vs legacy:
- Per-user RL now enforced at the gateway (same 10/min/IP budget).
- Rate-limit response omits Retry-After header; retryAfter is in the
  body per error-mapper.ts convention.
- ListScenarioTemplates emits affectedHs2: [] when the registry entry
  is null (all-sectors sentinel); proto repeated cannot carry null.
- RunScenario returns { jobId, status } (no statusUrl field — unused
  by SupplyChainPanel, drop from wire).

Gateway wiring:
- server/gateway.ts RPC_CACHE_TIER: list-scenario-templates → 'daily'
  (matches legacy max-age=3600); get-scenario-status → 'slow-browser'
  (premium short-circuit target, explicit entry required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: swap old run/status for the new
  run-scenario/get-scenario-status paths.
- api/scenario/v1/{run,status,templates}.ts deleted; 3 manifest
  exceptions removed (63 → 52 → 49 migration-pending).

Client:
- src/services/scenario/index.ts — typed client wrapper using
  premiumFetch (injects Clerk bearer / API key).
- src/components/SupplyChainPanel.ts — polling loop swapped from
  premiumFetch strings to runScenario/getScenarioStatus. Hard 20s
  timeout on run preserved via AbortSignal.any.

Tests:
- tests/scenario-handler.test.mjs — 18 new handler-level tests
  covering every security invariant + the worker envelope coercion.
- tests/edge-functions.test.mjs — scenario sections removed,
  replaced with a breadcrumb pointer to the new test file.

Docs: api-scenarios.mdx, scenario-engine.mdx, usage-rate-limits.mdx,
usage-errors.mdx, supply-chain.mdx refreshed with new paths.

Verified: typecheck, typecheck:api, lint:api-contract (49 entries),
lint:rate-limit-policies (6/180), lint:boundaries, route-cache-tier
(parity), full edge-functions (117) + scenario-handler (18).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(commit 8): migrate /api/v2/shipping/{route-intelligence,webhooks} → ShippingV2Service (#3207)

Partner-facing endpoints promoted to a typed sebuf service. Wire shape
preserved byte-for-byte (camelCase field names, ISO-8601 fetchedAt, the
same subscriberId/secret formats, the same SET + SADD + EXPIRE 30-day
Redis pipeline). Partner URLs /api/v2/shipping/* are unchanged.

RPCs landed:
- GET  /route-intelligence  → RouteIntelligence  (PRO, slow-browser)
- POST /webhooks            → RegisterWebhook    (PRO)
- GET  /webhooks            → ListWebhooks       (PRO, slow-browser)

The existing path-parameter URLs remain on the legacy edge-function
layout because sebuf's HTTP annotations don't currently model path
params (grep proto/**/*.proto for `path: "{…}"` returns zero). Those
endpoints are split into two Vercel dynamic-route files under
api/v2/shipping/webhooks/, behaviorally identical to the previous
hybrid file but cleanly separated:
- GET  /webhooks/{subscriberId}                → [subscriberId].ts
- POST /webhooks/{subscriberId}/rotate-secret  → [subscriberId]/[action].ts
- POST /webhooks/{subscriberId}/reactivate     → [subscriberId]/[action].ts

Both get manifest entries under `migration-pending` pointing at #3207.

Other changes
- scripts/enforce-sebuf-api-contract.mjs: extended GATEWAY_RE to accept
  api/v{N}/{domain}/[rpc].ts (version-first) alongside the canonical
  api/{domain}/v{N}/[rpc].ts; first-use of the reversed ordering is
  shipping/v2 because that's the partner contract.
- vite.config.ts: dev-server sebuf interceptor regex extended to match
  both layouts; shipping/v2 import + allRoutes entry added.
- server/gateway.ts: RPC_CACHE_TIER entries for /api/v2/shipping/
  route-intelligence + /webhooks (slow-browser; premium-gated endpoints
  short-circuit to slow-browser but the entries are required by
  tests/route-cache-tier.test.mjs).
- src/shared/premium-paths.ts: route-intelligence + webhooks added.
- tests/shipping-v2-handler.test.mjs: 18 handler-level tests covering
  PRO gate, iso2/cargoType/hs2 coercion, SSRF guards (http://, RFC1918,
  cloud metadata, IMDS), chokepoint whitelist, alertThreshold range,
  secret/subscriberId format, pipeline shape + 30-day TTL, cross-tenant
  owner isolation, `secret` omission from list response.

Manifest delta
- Removed: api/v2/shipping/route-intelligence.ts, api/v2/shipping/webhooks.ts
- Added:   api/v2/shipping/webhooks/[subscriberId].ts (migration-pending)
- Added:   api/v2/shipping/webhooks/[subscriberId]/[action].ts (migration-pending)
- Added:   api/internal/brief-why-matters.ts (internal-helper) — regression
  surface from the #3248 main merge, which introduced the file without a
  manifest entry. Filed here to keep the lint green; not strictly in scope
  for commit 8 but unblocking.

Net result: 49 → 47 `migration-pending` entries (one net-removal even
though webhook path-params stay pending, because two files collapsed
into two dynamic routes).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 1): SupplyChainServiceClient must use premiumFetch (#3207)

Signed-in browser pro users were silently hitting 401 on 8 supply-chain
premium endpoints (country-products, multi-sector-cost-shock,
country-chokepoint-index, bypass-options, country-cost-shock,
sector-dependency, route-explorer-lane, route-impact). The shared
client was constructed with globalThis.fetch, so no Clerk bearer or
X-WorldMonitor-Key was injected. The gateway's validateApiKey runs
with forceKey=true for PREMIUM_RPC_PATHS and 401s before isCallerPremium
is consulted. The generated client's try/catch collapses the 401 into
an empty-fallback return, leaving panels blank with no visible error.

Fix is one line at the client constructor: swap globalThis.fetch for
premiumFetch. The same pattern is already in use for insider-transactions,
stock-analysis, stock-backtest, scenario, trade (premiumClient) — this
was an omission on this client, not a new pattern.

premiumFetch no-ops safely when no credentials are available, so the
5 non-premium methods on this client (shippingRates, chokepointStatus,
chokepointHistory, criticalMinerals, shippingStress) continue to work
unchanged.

This also fixes two panels that were pre-existing latently broken on
main (chokepoint-index, bypass-options, etc. — predating #3207, not
regressions from it). Commit 6 expanded the surface by routing two more
methods through the same buggy client; this commit fixes the class.

From koala73 review (#3242 second-pass, HIGH new #1):
> Exact class PR #3233 fixed for RegionalIntelligenceBoard /
> DeductionPanel / trade / country-intel. Supply-chain was not in
> #3233's scope.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 2): restore 400 on input-shape errors for 2 supply-chain handlers (#3207)

Commit 6 collapsed all non-happy paths into empty-200 on
`get-country-products` and `get-multi-sector-cost-shock`, including
caller-bug cases that legacy returned 400 for:

- get-country-products: malformed iso2 → empty 200 (was 400)
- get-multi-sector-cost-shock: malformed iso2 / missing chokepointId /
  unknown chokepointId → empty 200 (was 400)

The commit message for 6 called out the 403-for-non-pro → empty-200
shift ("sebuf gateway pattern is empty-payload-on-deny") but not the
400 shift. They're different classes:

- Empty-payload-200 for PRO-deny: intentional contract change, already
  documented and applied across the service. Generated clients treat
  "you lack PRO" as "no data" — fine.
- Empty-payload-200 for malformed input: caller bug silently masked.
  External API consumers can't distinguish "bad wiring" from "genuinely
  no data", test harnesses lose the signal, bad calling code doesn't
  surface in Sentry.

Fix: `throw new ValidationError(violations)` on the 3 input-shape
branches. The generated sebuf server maps ValidationError → HTTP 400
(see src/generated/server/.../service_server.ts and leads/v1 which
already uses this pattern).

PRO-gate deny stays as empty-200 — that contract shift was intentional
and is preserved.

Regression tests added at tests/supply-chain-validation.test.mjs (8
cases) pinning the three-way contract:
- bad input                         → 400 (ValidationError)
- PRO-gate deny on valid input      → 200 empty
- valid PRO input, no data in Redis → 200 empty (unchanged)

From koala73 review (#3242 second-pass, HIGH new #2).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 3): restore statusUrl on RunScenarioResponse + document 202→200 wire break (#3207)

Commit 7 silently shifted /api/scenario/v1/run-scenario's response
contract in two ways that the commit message covered only partially:

1. HTTP 202 Accepted → HTTP 200 OK
2. Dropped `statusUrl` string from the response body

The `statusUrl` drop was mentioned as "unused by SupplyChainPanel" but
not framed as a contract change. The 202 → 200 shift was not mentioned
at all. This is a same-version (v1 → v1) migration, so external callers
that key off either signal — `response.status === 202` or
`response.body.statusUrl` — silently branch incorrectly.

Evaluated options:
  (a) sebuf per-RPC status-code config — not available. sebuf's
      HttpConfig only models `path` and `method`; no status annotation.
  (b) Bump to scenario/v2 — judged heavier than the break itself for
      a single status-code shift. No in-repo caller uses 202 or
      statusUrl; the docs-level impact is containable.
  (c) Accept the break, document explicitly, partially restore.

Took option (c):

- Restored `statusUrl` in the proto (new field `string status_url = 3`
  on RunScenarioResponse). Server computes
  `/api/scenario/v1/get-scenario-status?jobId=<encoded job_id>` and
  populates it on every successful enqueue. External callers that
  followed this URL keep working unchanged.
- 202 → 200 is not recoverable inside the sebuf generator, so it is
  called out explicitly in two places:
    - docs/api-scenarios.mdx now includes a prominent `<Warning>` block
      documenting the v1→v1 contract shift + the suggested migration
      (branch on response body shape, not HTTP status).
    - RunScenarioResponse proto comment explains why 200 is the new
      success status on enqueue.
  OpenAPI bundle regenerated to reflect the restored statusUrl field.

- Regression test added in tests/scenario-handler.test.mjs pinning
  `statusUrl` to the exact URL-encoded shape — locks the invariant so
  a future proto rename or handler refactor can't silently drop it
  again.

From koala73 review (#3242 second-pass, HIGH new #3).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 1/2): close webhook tenant-isolation gap on shipping/v2 (#3207)

Koala flagged this as a merge blocker in PR #3242 review.

server/worldmonitor/shipping/v2/{register-webhook,list-webhooks}.ts
migrated without reinstating validateApiKey(req, { forceKey: true }),
diverging from both the sibling api/v2/shipping/webhooks/[subscriberId]
routes and the documented "X-WorldMonitor-Key required" contract in
docs/api-shipping-v2.mdx.

Attack surface: the gateway accepts Clerk bearer auth as a pro signal.
A Clerk-authenticated pro user with no X-WorldMonitor-Key reaches the
handler, callerFingerprint() falls back to 'anon', and every such
caller collapses into a shared webhook:owner:anon:v1 bucket. The
defense-in-depth ownerTag !== ownerHash check in list-webhooks.ts
doesn't catch it because both sides equal 'anon' — every Clerk-session
holder could enumerate / overwrite every other Clerk-session pro
tenant's registered webhook URLs.

Fix: reinstate validateApiKey(ctx.request, { forceKey: true }) at the
top of each handler, throwing ApiError(401) when absent. Matches the
sibling routes exactly and the published partner contract.

Tests:
- tests/shipping-v2-handler.test.mjs: two existing "non-PRO → 403"
  tests for register/list were using makeCtx() with no key, which now
  fails at the 401 layer first. Renamed to "no API key → 401
  (tenant-isolation gate)" with a comment explaining the failure mode
  being tested. 18/18 pass.

Verified: typecheck:api, lint:api-contract (no change), lint:boundaries,
lint:rate-limit-policies, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(review HIGH 2/2): restore v1 path aliases on scenario + supply-chain (#3207)

Koala flagged this as a merge blocker in PR #3242 review.

Commits 6 + 7 of #3207 renamed five documented v1 URLs to the sebuf
method-derived paths and deleted the legacy edge-function files:

  POST /api/scenario/v1/run                       → run-scenario
  GET  /api/scenario/v1/status                    → get-scenario-status
  GET  /api/scenario/v1/templates                 → list-scenario-templates
  GET  /api/supply-chain/v1/country-products      → get-country-products
  GET  /api/supply-chain/v1/multi-sector-cost-shock → get-multi-sector-cost-shock

server/router.ts is an exact static-match table (Map keyed on `METHOD
PATH`), so any external caller — docs, partner scripts, grep-the-
internet — hitting the old documented URL would 404 on first request
after merge. Commit 8 (shipping/v2) preserved partner URLs byte-for-
byte; the scenario + supply-chain renames missed that discipline.

Fix: add five thin alias edge functions that rewrite the pathname to
the canonical sebuf path and delegate to the domain [rpc].ts gateway
via a new server/alias-rewrite.ts helper. Premium gating, rate limits,
entitlement checks, and cache-tier lookups all fire on the canonical
path — aliases are pure URL rewrites, not a duplicate handler pipeline.

  api/scenario/v1/{run,status,templates}.ts
  api/supply-chain/v1/{country-products,multi-sector-cost-shock}.ts

Vite dev parity: file-based routing at api/ is a Vercel concern, so the
dev middleware (vite.config.ts) gets a matching V1_ALIASES rewrite map
before the router dispatch.

Manifest: 5 new entries under `deferred` with removal_issue=#3282
(tracking their retirement at the next v1→v2 break). lint:api-contract
stays green (89 files checked, 55 manifest entries validated).

Docs:
- docs/api-scenarios.mdx: migration callout at the top with the full
  old→new URL table and a link to the retirement issue.
- CHANGELOG.md + docs/changelog.mdx: Changed entry documenting the
  rename + alias compat + the 202→200 shift (from commit 23c821a1).

Verified: typecheck:api, lint:api-contract, lint:rate-limit-policies,
lint:boundaries, test:data (6005/6005).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-04-22 09:55:59 +03:00
Elie Habib
01f6057389 feat(simulation): MiroFish Phase 2 — theater-limited simulation runner (#2220) 
* feat(simulation): MiroFish Phase 2 — theater-limited simulation runner

Adds the simulation execution layer that consumes simulation-package.json
and produces simulation-outcome.json for maritime chokepoint + energy/logistics
theaters, closing the WorldMonitor → MiroFish handoff loop.

Changes:
- scripts/seed-forecasts.mjs: 2-round LLM simulation runner (prompt builders,
  JSON extractor, runTheaterSimulation, writeSimulationOutcome, task queue
  with NX dedup lock, runSimulationWorker poll loop)
- scripts/process-simulation-tasks.mjs: standalone worker entry point
- proto: GetSimulationOutcome RPC + make generate
- server/worldmonitor/forecast/v1/get-simulation-outcome.ts: RPC handler
- server/gateway.ts: slow tier for get-simulation-outcome
- api/health.js: simulationOutcomeLatest in STANDALONE + ON_DEMAND keys
- tests: 14 new tests for simulation runner functions

* fix(simulation): address P1/P2 code review findings from PR #2220

Security (P1 #018):
- sanitizeForPrompt() applied to all entity/seed fields interpolated into
  Round 1 prompt (entityId, class, stance, seedId, type, timing)
- sanitizeForPrompt() applied to actorId and entityIds in Round 2 prompt
- sanitizeForPrompt() + length caps applied to all LLM array fields written
  to R2 (dominantReactions, stabilizers, invalidators, keyActors, timingMarkers)

Validation (P1 #019):
- Added validateRunId() regex guard
- Applied in enqueueSimulationTask() and processNextSimulationTask() loop

Type safety (P1 #020):
- Added isOutcomePointer() and isPackagePointer() type guards in TS handlers
- Replaced unsafe as-casts with runtime-validated guards in both handlers

Correctness (P2 #022):
- Log warning when pkgPointer.runId does not match task runId

Architecture (P2 #024):
- isMaritimeChokeEnergyCandidate() accepts both flat and nested topBucketId
- Call site simplified to pass theater directly

Performance (P2 #025):
- SIMULATION_ROUND1_MAX_TOKENS raised 1800 to 2200
- Added max 3 initialReactions instruction to Round 1 prompt

Maintainability (P2 #026):
- Simulation pointer keys exported from server/_shared/cache-keys.ts
- Both TS handlers import from shared location

Documentation (P2 #027):
- Strengthened runId no-op description in proto and OpenAPI spec

* fix(todos): add blank lines around lists in markdown todo files

* style(api): reformat openapi yaml to match linter output

* test(simulation): add flat-shape filter test + getSimulationOutcome handler coverage

Two tests identified as missing during PR #2220 review:

1. isMaritimeChokeEnergyCandidate flat-shape tests — covers the || candidate.topBucketId
   normalization added in the P1/P2 review pass. The existing tests only used the nested
   marketContext.topBucketId shape; this adds the flat root-field shape that arrives from
   the simulation-package.json JSON (selectedTheaters entries have topBucketId at root).

2. getSimulationOutcome handler structural tests — verifies the isOutcomePointer guard,
   found:false NOT_FOUND return, found:true success path, note population on runId mismatch,
   and redis_unavailable error string. Follows the readSrc static-analysis pattern used
   elsewhere in server-handlers.test.mjs (handler imports Redis so full integration test
   would require a test Redis instance).
 2026-03-25 13:55:59 +04:00
lspassos1
1886161587 fix: resilient assertions and test stability (#1389) 
Co-authored-by: Elie Habib <elie.habib@gmail.com>
 2026-03-12 00:19:16 +04:00
Elie Habib
363cf5e71c perf(api): convert POST RPCs to GET for CDN caching (#795) 
* perf(api): convert classify-event to GET and add summarize-article cache endpoint for CDN caching

classify-event (7.9M calls/wk) was POST — bypassing all CDN caching.
Converting to GET with static cache tier (1hr) enables Cloudflare edge
caching. Degraded responses (no API key, empty title, errors) are marked
no-cache to prevent caching error states.

summarize-article has repeated headlines too large for URL params. Added
a new GetSummarizeArticleCache GET endpoint that looks up Redis by a
deterministic cache key. Client computes key via shared
buildSummaryCacheKey(), tries GET first (CDN-cacheable), falls back to
existing POST on miss. Shared module ensures client/server key parity.

* fix(types): wire missing DeductSituation and ListGulfQuotes RPCs, fix tsc errors

- Added DeductSituation RPC to intelligence/v1/service.proto (messages
  existed, RPC declaration was missing)
- Added ListGulfQuotes proto + RPC to market/v1/service.proto (handler
  existed, proto was missing)
- Fixed scrapedAt type mismatch in conflict/index.ts (int64 → string)
- Added @ts-nocheck to generated files with codegen type bugs
- Regenerated all sebuf client/server code

* fix(types): fix int64→string type mismatch in list-iran-events.ts
 2026-03-02 22:01:32 +04:00
Sebastien Melki
c939cc6296 Proto-first API rebuild: sebuf contracts, handlers, gateway, and generated docs (#106) 
* docs: initialize sebuf integration project with codebase map

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: add project config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: complete project research

* docs: define v1 requirements (34 requirements, 8 categories)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: create roadmap (8 phases)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(01): capture phase context

* docs(state): record phase 1 context session

* docs(01): research phase domain - buf toolchain, sebuf codegen, proto patterns

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(01-proto-foundation): create phase plan

* chore(01-01): configure buf toolchain with buf.yaml, buf.gen.yaml, buf.lock

- buf.yaml v2 with STANDARD+COMMENTS lint, FILE+PACKAGE+WIRE_JSON breaking, deps on protovalidate and sebuf
- buf.gen.yaml configures protoc-gen-ts-client, protoc-gen-ts-server, protoc-gen-openapiv3 plugins
- buf.lock generated with resolved dependency versions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(01-01): add shared core proto type definitions

- geo.proto: GeoCoordinates with lat/lng validation, BoundingBox for spatial queries
- time.proto: TimeRange with google.protobuf.Timestamp start/end
- pagination.proto: cursor-based PaginationRequest (1-100 page_size) and PaginationResponse
- i18n.proto: LocalizableString for pre-localized upstream API strings
- identifiers.proto: typed ID wrappers (HotspotID, EventID, ProviderID) for cross-domain refs
- general_error.proto: GeneralError with RateLimited, UpstreamDown, GeoBlocked, MaintenanceMode

All files pass buf lint (STANDARD+COMMENTS) and buf build with zero errors.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(01-01): complete buf toolchain and core proto types plan

- SUMMARY.md documents 2 tasks, 9 files created, 2 deviations auto-fixed
- STATE.md updated: plan 1/2 in phase 1, decisions recorded
- ROADMAP.md updated: phase 01 in progress (1/2 plans)
- REQUIREMENTS.md updated: PROTO-01, PROTO-02, PROTO-03 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor(01-01): use int64 epoch millis instead of google.protobuf.Timestamp

User preference: all time fields use int64 (Unix epoch milliseconds)
instead of google.protobuf.Timestamp for simpler serialization and
JS interop. Applied to TimeRange and MaintenanceMode.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(01-02): create test domain proto files with core type imports

- Add test_item.proto with GeoCoordinates import and int64 timestamps
- Add get_test_items.proto with TimeRange and Pagination imports
- Add service.proto with HTTP annotations for TestService
- All proto files pass buf lint and buf build

* feat(01-02): run buf generate and create Makefile for code generation pipeline

- Add Makefile with generate, lint, clean, install, check, format, breaking targets
- Update buf.gen.yaml with managed mode and paths=source_relative for correct output paths
- Generate TypeScript client (TestServiceClient class) at src/generated/client/
- Generate TypeScript server (TestServiceHandler interface) at src/generated/server/
- Generate OpenAPI 3.1.0 specs (JSON + YAML) at docs/api/
- Core type imports (GeoCoordinates, TimeRange, Pagination) flow through to generated output

* docs(01-02): complete test domain code generation pipeline plan

- Create 01-02-SUMMARY.md with pipeline validation results
- Update STATE.md: phase 1 complete, 2/2 plans done, new decisions recorded
- Update ROADMAP.md: phase 1 marked complete (2/2)
- Update REQUIREMENTS.md: mark PROTO-04 and PROTO-05 complete

* docs(phase-01): complete phase execution and verification

* test(01): complete UAT - 6 passed, 0 issues

* feat(2A): define all 17 domain proto packages with generated clients, servers, and OpenAPI specs

Remove test domain protos (Phase 1 scaffolding). Add core enhancements
(severity.proto, country.proto, expanded identifiers.proto). Define all
17 domain services: seismology, wildfire, climate, conflict, displacement,
unrest, military, aviation, maritime, cyber, market, prediction, economic,
news, research, infrastructure, intelligence. 79 proto files producing
34 TypeScript files and 34 OpenAPI specs. buf lint clean, tsc clean.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2B): add server runtime phase context and handoff checkpoint

Prepare Phase 2B with full context file covering deliverables,
key reference files, generated code patterns, and constraints.
Update STATE.md with resume pointer.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2B): research phase domain

* docs(2B): create phase plan

* feat(02-01): add shared server infrastructure (router, CORS, error mapper)

- router.ts: Map-based route matcher from RouteDescriptor[] arrays
- cors.ts: TypeScript port of api/_cors.js with POST/OPTIONS methods
- error-mapper.ts: onError callback handling ApiError, network, and unknown errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(02-01): implement seismology handler as first end-to-end proof

- Implements SeismologyServiceHandler from generated server types
- Fetches USGS M4.5+ earthquake GeoJSON feed and transforms to proto-shaped Earthquake[]
- Maps all fields: id, place, magnitude, depthKm, location, occurredAt (String), sourceUrl

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(02-01): complete server infrastructure plan

- SUMMARY.md with task commits, decisions, and self-check
- STATE.md updated: position, decisions, session info
- REQUIREMENTS.md: SERVER-01, SERVER-02, SERVER-06 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(02-02): create Vercel catch-all gateway, tsconfig.api.json, and typecheck:api script

- api/[[...path]].ts mounts seismology routes via catch-all with CORS on every response path
- tsconfig.api.json extends base config without vite/client types for edge runtime
- package.json adds typecheck:api script

* feat(02-02): add Vite dev server plugin for sebuf API routes

- sebufApiPlugin() intercepts /api/{domain}/v1/* in dev mode
- Uses dynamic imports to lazily load handler modules inside configureServer
- Converts Connect IncomingMessage to Web Standard Request
- CORS headers applied to all plugin responses (200, 204, 403, 404)
- Falls through to existing proxy rules for non-sebuf /api/* paths

* docs(02-02): complete gateway integration plan

- SUMMARY.md documenting catch-all gateway + Vite plugin implementation
- STATE.md updated: Phase 2B complete, decisions recorded
- ROADMAP.md updated: Phase 02 marked complete (2/2 plans)
- REQUIREMENTS.md: SERVER-03, SERVER-04, SERVER-05 marked complete

* docs(02-server-runtime): create gap closure plan for SERVER-05 Tauri sidecar

* feat(02-03): add esbuild compilation step for sebuf sidecar gateway bundle

- Create scripts/build-sidecar-sebuf.mjs that bundles api/[[...path]].ts into a single ESM .js file
- Add build:sidecar-sebuf npm script and chain it into the main build command
- Install esbuild as explicit devDependency
- Gitignore the compiled api/[[...path]].js build artifact

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(02-03): verify sidecar discovery and annotate SERVER-05 gap closure

- Confirm compiled bundle handler returns status 200 for POST requests
- Add gap closure note to SERVER-05 in REQUIREMENTS.md
- Verify typecheck:api and full build pipeline pass without regressions

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(02-03): complete sidecar sebuf bundle plan

- Create 02-03-SUMMARY.md documenting esbuild bundle compilation
- Update STATE.md with plan 03 position, decisions, and metrics
- Update ROADMAP.md plan progress (3/3 plans complete)
- Annotate SERVER-05 gap closure in REQUIREMENTS.md

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-02): complete phase execution

* docs(2C): capture seismology migration phase context

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(state): record phase 2C context session

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2C): research seismology migration phase

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2C): create seismology migration phase plan

* feat(2C-01): annotate all int64 time fields with INT64_ENCODING_NUMBER

- Vendor sebuf/http/annotations.proto locally with Int64Encoding extension (50010)
- Remove buf.build/sebmelki/sebuf BSR dep, use local vendored proto instead
- Add INT64_ENCODING_NUMBER annotation to 34 time fields across 20 proto files
- Regenerate all TypeScript client and server code (time fields now `number` not `string`)
- Fix seismology handler: occurredAt returns number directly (no String() wrapper)
- All non-time int64 fields (displacement counts, population) left as string
- buf lint, buf generate, tsc, and sidecar build all pass with zero errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2C-01): complete INT64_ENCODING_NUMBER plan

- Create 2C-01-SUMMARY.md with execution results and deviations
- Update STATE.md: plan 01 complete, int64 blocker resolved, new decisions
- Update ROADMAP.md: mark 2C-01 plan complete
- Update REQUIREMENTS.md: mark CLIENT-01 complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(lint): exclude .planning/ from markdownlint

GSD planning docs use formatting that triggers MD032 -- these are
machine-generated and not user-facing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2C-02): rewrite earthquake adapter to use SeismologyServiceClient and adapt all consumers to proto types

- Replace legacy fetch/circuit-breaker adapter with port/adapter wrapping SeismologyServiceClient
- Update 7 consuming files to import Earthquake from @/services/earthquakes (the port)
- Adapt all field accesses: lat/lon -> location?.latitude/longitude, depth -> depthKm, time -> occurredAt, url -> sourceUrl
- Remove unused filterByTime from Map.ts (only called for earthquakes, replaced with inline filter)
- Update e2e test data to proto Earthquake shape

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(2C-02): delete legacy earthquake endpoint, remove Vite proxy, clean API_URLS config

- Delete api/earthquakes.js (legacy Vercel edge function proxying USGS)
- Remove /api/earthquake Vite dev proxy (sebufApiPlugin handles seismology now)
- Remove API_URLS.earthquakes entry from base config (no longer referenced)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2C-02): complete seismology client wiring plan

- Create 2C-02-SUMMARY.md with execution results
- Update STATE.md: phase 2C complete, decisions, metrics
- Update ROADMAP.md: mark 2C-02 and phase 2C complete
- Mark requirements CLIENT-02, CLIENT-04, CLEAN-01, CLEAN-02 complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2C): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2D): create wildfire migration phase plan

* feat(2D-01): enhance FireDetection proto and implement wildfire handler

- Add region (field 8) and day_night (field 9) to FireDetection proto
- Regenerate TypeScript client and server types
- Implement WildfireServiceHandler with NASA FIRMS CSV proxy
- Fetch all 9 monitored regions in parallel via Promise.allSettled
- Graceful degradation to empty list when API key is missing

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2D-01): wire wildfire routes into gateway and rebuild sidecar

- Import createWildfireServiceRoutes and wildfireHandler in catch-all
- Mount wildfire routes alongside seismology in allRoutes array
- Rebuild sidecar-sebuf bundle with wildfire endpoint included

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2D-01): complete wildfire handler plan

- Create 2D-01-SUMMARY.md with execution results
- Update STATE.md position to 2D plan 01 complete
- Update ROADMAP.md with 2D progress (1/2 plans)
- Mark DOMAIN-01 requirement complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2D-02): create wildfires service module and rewire all consumers

- Add src/services/wildfires/index.ts with fetchAllFires, computeRegionStats, flattenFires, toMapFires
- Rewire App.ts to import from @/services/wildfires with proto field mappings
- Rewire SatelliteFiresPanel.ts to import FireRegionStats from @/services/wildfires
- Update signal-aggregator.ts source comment

* chore(2D-02): delete legacy wildfire endpoint and service module

- Remove api/firms-fires.js (replaced by api/server/worldmonitor/wildfire/v1/handler.ts)
- Remove src/services/firms-satellite.ts (replaced by src/services/wildfires/index.ts)
- Zero dangling references confirmed
- Full build passes (tsc, vite, sidecar)

* docs(2D-02): complete wildfire consumer wiring plan

- Create 2D-02-SUMMARY.md with execution results
- Update STATE.md: phase 2D complete, progress ~52%
- Update ROADMAP.md: phase 2D plan progress

* docs(phase-2D): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2E): research climate migration domain

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2E): create phase plan

* feat(2E-01): implement climate handler with 15-zone monitoring and baseline comparison

- Create ClimateServiceHandler with 15 hardcoded monitored zones matching legacy
- Parallel fetch from Open-Meteo Archive API via Promise.allSettled
- 30-day baseline comparison: last 7 days vs preceding baseline
- Null filtering with paired data points, minimum 14-point threshold
- Severity classification (normal/moderate/extreme) and type (warm/cold/wet/dry/mixed)
- 1-decimal rounding for tempDelta and precipDelta
- Proto ClimateAnomaly mapping with GeoCoordinates

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2E-01): wire climate routes into gateway and rebuild sidecar

- Import createClimateServiceRoutes and climateHandler in catch-all gateway
- Mount climate routes alongside seismology and wildfire
- Rebuild sidecar-sebuf bundle with climate routes included

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2E-01): complete climate handler plan

- Create 2E-01-SUMMARY.md with execution results
- Update STATE.md: position to 2E plan 01, add decisions
- Update ROADMAP.md: mark 2E-01 complete, update progress table

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2E-02): rewrite climate service module and rewire all consumers

- Replace src/services/climate.ts with src/services/climate/index.ts directory module
- Port/adapter pattern: ClimateServiceClient maps proto shapes to legacy consumer shapes
- Rewire ClimateAnomalyPanel, DeckGLMap, MapContainer, country-instability, conflict-impact
- All 6 consumers import ClimateAnomaly from @/services/climate instead of @/types
- Drop dead getSeverityColor function, keep getSeverityIcon and formatDelta
- Fix minSeverity required param in listClimateAnomalies call

* chore(2E-02): delete legacy climate endpoint and remove dead types

- Delete api/climate-anomalies.js (replaced by sebuf climate handler)
- Remove ClimateAnomaly and AnomalySeverity from src/types/index.ts
- Full build passes with zero errors

* docs(2E-02): complete climate client wiring plan

- Create 2E-02-SUMMARY.md with execution results
- Update STATE.md: phase 2E complete, decisions, session continuity
- Update ROADMAP.md: phase 2E progress

* docs(phase-2E): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2F): research prediction migration domain

* docs(2F): create prediction migration phase plans

* feat(2F-01): implement prediction handler with Gamma API proxy

- PredictionServiceHandler proxying Gamma API with 8s timeout
- Maps events/markets to proto PredictionMarket with 0-1 yesPrice scale
- Graceful degradation: returns empty markets on any failure (Cloudflare expected)
- Supports category-based events endpoint and default markets endpoint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2F-01): wire prediction routes into gateway

- Import createPredictionServiceRoutes and predictionHandler
- Mount prediction routes in allRoutes alongside seismology, wildfire, climate
- Sidecar bundle rebuilt successfully (21.2 KB)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2F-01): complete prediction handler plan

- SUMMARY.md with handler implementation details and deviation log
- STATE.md updated to 2F in-progress position with decisions
- ROADMAP.md updated to 1/2 plans complete for phase 2F
- REQUIREMENTS.md marked DOMAIN-02 complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2F-02): create prediction service module and rewire all consumers

- Create src/services/prediction/index.ts preserving all business logic from polymarket.ts
- Replace strategy 4 (Vercel edge) with PredictionServiceClient in polyFetch
- Update barrel export from polymarket to prediction in services/index.ts
- Rewire 7 consumers to import PredictionMarket from @/services/prediction
- Fix 3 yesPrice bugs: CountryIntelModal (*100), App.ts search (*100), App.ts snapshot (1-y)
- Drop dead code getPolymarketStatus()

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(2F-02): delete legacy endpoint and remove dead types

- Delete api/polymarket.js (replaced by sebuf handler)
- Delete src/services/polymarket.ts (replaced by src/services/prediction/index.ts)
- Remove PredictionMarket interface from src/types/index.ts (now in prediction module)
- Type check and sidecar build both pass

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2F-02): complete prediction consumer wiring plan

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2F): complete phase execution

* docs(phase-2F): fix roadmap plan counts and completion status

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2G): research displacement migration phase

* docs(2G): create displacement migration phase plans

* feat(2G-01): implement displacement handler with UNHCR API pagination and aggregation

- 40-entry COUNTRY_CENTROIDS map for geographic coordinates
- UNHCR Population API pagination (10,000/page, 25-page guard)
- Year fallback: current year to current-2 until data found
- Per-country origin + asylum aggregation with unified merge
- Global totals computation across all raw records
- Flow corridor building sorted by refugees, capped by flowLimit
- All int64 fields returned as String() per proto types
- Graceful empty response on any failure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2G-01): wire displacement routes into gateway and rebuild sidecar

- Import createDisplacementServiceRoutes and displacementHandler
- Mount displacement routes alongside seismology, wildfire, climate, prediction
- Sidecar bundle rebuilt with displacement included (31.0 KB)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2G-01): complete displacement handler plan

- SUMMARY.md with execution metrics and decisions
- STATE.md updated to 2G phase position
- ROADMAP.md updated with 2G-01 plan progress

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2G-02): create displacement service module and rewire all consumers

- Create src/services/displacement/index.ts as port/adapter using DisplacementServiceClient
- Map proto int64 strings to numbers and GeoCoordinates to flat lat/lon
- Preserve circuit breaker, presentation helpers (getDisplacementColor, formatPopulation, etc.)
- Rewire App.ts, DisplacementPanel, MapContainer, DeckGLMap, conflict-impact, country-instability
- Delete legacy src/services/unhcr.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(2G-02): delete legacy endpoint and remove dead displacement types

- Delete api/unhcr-population.js (replaced by displacement handler from 2G-01)
- Remove DisplacementFlow, CountryDisplacement, UnhcrSummary from src/types/index.ts
- All consumers now import from @/services/displacement
- Sidecar rebuild, tsc, and full Vite build pass clean

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2G-02): complete displacement consumer wiring plan

- SUMMARY.md with 2 task commits, decisions, deviation documentation
- STATE.md updated: phase 2G complete, 02/02 plans done
- ROADMAP.md updated with plan progress

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2G): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2H): research aviation migration phase

* docs(2H): create aviation migration phase plans

* feat(2H-01): implement aviation handler with FAA XML parsing and simulated delays

- Install fast-xml-parser for server-side XML parsing (edge-compatible)
- Create AviationServiceHandler with FAA NASSTATUS XML fetch and parse
- Enrich US airports with MONITORED_AIRPORTS metadata (lat, lon, name, icao)
- Generate simulated delays for non-US airports with rush-hour weighting
- Map short-form strings to proto enums (FlightDelayType, FlightDelaySeverity, etc.)
- Wrap flat lat/lon into GeoCoordinates for proto response
- Graceful empty alerts on any upstream failure

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2H-01): wire aviation routes into gateway and rebuild sidecar

- Mount createAviationServiceRoutes in catch-all gateway alongside 5 existing domains
- Import aviationHandler for route wiring
- Rebuild sidecar-sebuf bundle with aviation routes included

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2H-01): complete aviation handler plan

- Create 2H-01-SUMMARY.md with execution results
- Update STATE.md position to 2H-01 with aviation decisions
- Update ROADMAP.md progress for phase 2H (1/2 plans)
- Mark DOMAIN-08 requirement as complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2H-02): create aviation service module and rewire all consumers

- Create src/services/aviation/index.ts as port/adapter wrapping AviationServiceClient
- Map proto enum strings to short-form (severity, delayType, region, source)
- Unwrap GeoCoordinates to flat lat/lon, convert epoch-ms updatedAt to Date
- Preserve circuit breaker with identical name string
- Rewire Map, DeckGLMap, MapContainer, MapPopup, map-harness to import from @/services/aviation
- Update barrel export: flights -> aviation
- Delete legacy src/services/flights.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(2H-02): delete legacy endpoint and remove dead aviation types

- Delete api/faa-status.js (replaced by aviation handler in 2H-01)
- Remove FlightDelaySource, FlightDelaySeverity, FlightDelayType, AirportRegion, AirportDelayAlert from src/types/index.ts
- Preserve MonitoredAirport with inlined region type union
- Full build (tsc + vite + sidecar) passes with zero errors

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2H-02): complete aviation consumer wiring plan

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2H): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2I): research phase domain

* docs(2I): create phase plan

* feat(2I-01): implement ResearchServiceHandler with 3 RPCs

- arXiv XML parsing with fast-xml-parser (ignoreAttributes: false for attributes)
- GitHub trending repos with primary + fallback API URLs
- Hacker News Firebase API with 2-step fetch and bounded concurrency (10)
- All RPCs return empty arrays on failure (graceful degradation)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2I-01): mount research routes in gateway and rebuild sidecar

- Import createResearchServiceRoutes and researchHandler in catch-all gateway
- Add research routes to allRoutes array (after aviation)
- Sidecar bundle rebuilt (116.6 KB)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2I-01): complete research handler plan

- SUMMARY.md with self-check passed
- STATE.md updated to phase 2I, plan 01 of 02
- ROADMAP.md updated with plan 2I-01 complete
- REQUIREMENTS.md: DOMAIN-05 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2I-02): create research service module and delete legacy code

- Add src/services/research/index.ts with fetchArxivPapers, fetchTrendingRepos, fetchHackernewsItems backed by ResearchServiceClient
- Re-export proto types ArxivPaper, GithubRepo, HackernewsItem (no enum mapping needed)
- Circuit breakers wrap all 3 client calls with empty-array fallback
- Delete legacy API endpoints: api/arxiv.js, api/github-trending.js, api/hackernews.js
- Delete legacy service files: src/services/arxiv.ts, src/services/github-trending.ts, src/services/hackernews.ts
- Remove arxiv, githubTrending, hackernews entries from API_URLS and REFRESH_INTERVALS in config

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2I-02): complete research consumer wiring plan

- SUMMARY.md documenting service module creation and 6 legacy file deletions
- STATE.md updated: phase 2I complete, decisions recorded
- ROADMAP.md updated: phase 2I marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2I): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2J): complete unrest migration research

* docs(2J): create unrest migration phase plans

* feat(2J-01): implement UnrestServiceHandler with ACLED + GDELT dual-fetch

- Create handler with listUnrestEvents RPC proxying ACLED API and GDELT GEO API
- ACLED fetch uses Bearer auth from ACLED_ACCESS_TOKEN env var, returns empty on missing token
- GDELT fetch returns GeoJSON protest events with no auth required
- Deduplication uses 0.5-degree grid + date key, preferring ACLED over GDELT on collision
- Severity classification and event type mapping ported from legacy protests.ts
- Sort by severity (high first) then recency (newest first)
- Graceful degradation: returns empty events on any upstream failure

* feat(2J-01): mount unrest routes in gateway and rebuild sidecar

- Import createUnrestServiceRoutes and unrestHandler in catch-all gateway
- Add unrest service routes to allRoutes array
- Sidecar bundle rebuilt to include unrest endpoint
- RPC routable at POST /api/unrest/v1/list-unrest-events

* docs(2J-01): complete unrest handler plan

- Create 2J-01-SUMMARY.md with execution results and self-check
- Update STATE.md with phase 2J position, decisions, session continuity
- Update ROADMAP.md with plan 01 completion status

* feat(2J-02): create unrest service module with proto-to-legacy type mapping

- Full adapter maps proto UnrestEvent to legacy SocialUnrestEvent shape
- 4 enum mappers: severity, eventType, sourceType, confidence
- fetchProtestEvents returns ProtestData with events, byCountry, highSeverityCount, sources
- getProtestStatus infers ACLED configuration from response event sources
- Circuit breaker wraps client call with empty fallback

* feat(2J-02): update services barrel, remove vite proxies, delete legacy files

- Services barrel: protests -> unrest re-export
- Vite proxy entries removed: /api/acled, /api/gdelt-geo
- Legacy files deleted: api/acled.js, api/gdelt-geo.js, src/services/protests.ts
- Preserved: api/acled-conflict.js (conflict domain), SocialUnrestEvent type

* docs(2J-02): complete unrest service module plan

- SUMMARY.md created with full adapter pattern documentation
- STATE.md updated: 2J-02 complete, decisions recorded
- ROADMAP.md updated: Phase 2J marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2J): complete phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2K): complete conflict migration research

* docs(2K): create phase plan

* feat(2K-01): implement ConflictServiceHandler with 3 RPCs

- listAcledEvents proxies ACLED API for battles/explosions/violence with Bearer auth
- listUcdpEvents discovers UCDP GED API version dynamically, fetches backward with 365-day trailing window
- getHumanitarianSummary proxies HAPI API with ISO-2 to ISO-3 country mapping
- All RPCs have graceful degradation returning empty on failure

* feat(2K-01): mount conflict routes in gateway and rebuild sidecar

- Add createConflictServiceRoutes and conflictHandler imports to catch-all gateway
- Spread conflict routes into allRoutes array (3 RPC endpoints)
- Rebuild sidecar bundle with conflict endpoints included

* docs(2K-01): complete conflict handler plan

- Create 2K-01-SUMMARY.md with execution details and self-check
- Update STATE.md: position to 2K-01, add 5 decisions
- Update ROADMAP.md: mark 2K-01 complete (1/2 plans done)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2K-02): create conflict service module with 4-shape proto-to-legacy type mapping

- Port/adapter mapping AcledConflictEvent -> ConflictEvent, UcdpViolenceEvent -> UcdpGeoEvent, HumanitarianCountrySummary -> HapiConflictSummary
- UCDP classifications derived heuristically from GED events (deaths/events thresholds -> war/minor/none)
- deduplicateAgainstAcled ported exactly with haversine + date + fatality matching
- 3 circuit breakers for 3 RPCs, exports 5 functions + 2 group helpers + all legacy types

* feat(2K-02): rewire consumer imports and delete 9 legacy conflict files

- App.ts consolidated from 4 direct imports to single @/services/conflict import
- country-instability.ts consolidated from 3 type imports to single ./conflict import
- Deleted 4 API endpoints: acled-conflict.js, ucdp-events.js, ucdp.js, hapi.js
- Deleted 4 service files: conflicts.ts, ucdp.ts, ucdp-events.ts, hapi.ts
- Deleted 1 dead code file: conflict-impact.ts
- UcdpGeoEvent preserved in src/types/index.ts (scope guard for map components)

* docs(2K-02): complete conflict service module plan

- SUMMARY.md with 4-shape proto adapter, consumer consolidation, 9 legacy deletions
- STATE.md updated: Phase 2K complete (2/2 plans), progress ~100%
- ROADMAP.md updated: Phase 2K marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-2K): complete conflict migration phase execution

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2L): research maritime migration phase domain

* docs(2L): create maritime migration phase plans

* feat(2L-01): implement MaritimeServiceHandler with 2 RPCs

- getVesselSnapshot proxies WS relay with wss->https URL conversion
- Maps density/disruptions to proto shape with GeoCoordinates nesting
- Disruption type/severity mapped from lowercase to proto enums
- listNavigationalWarnings proxies NGA MSI broadcast warnings API
- NGA military date parsing (081653Z MAY 2024) to epoch ms
- Both RPCs gracefully degrade to empty on upstream failure
- No caching (client-side polling manages refresh intervals)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2L-01): mount maritime routes in gateway and rebuild sidecar

- Import createMaritimeServiceRoutes and maritimeHandler
- Add maritime routes to allRoutes array in catch-all gateway
- Sidecar bundle rebuilt (148.0 KB) with maritime endpoints
- RPCs routable at /api/maritime/v1/get-vessel-snapshot and /api/maritime/v1/list-navigational-warnings

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(2L-01): complete maritime handler plan

- SUMMARY.md with 2 task commits documented
- STATE.md updated to 2L phase, plan 01/02 complete
- ROADMAP.md progress updated for phase 2L
- REQUIREMENTS.md: DOMAIN-06 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(2L-02): create maritime service module with hybrid fetch and polling/callback preservation

- Port/adapter wrapping MaritimeServiceClient for proto RPC path
- Full polling/callback architecture preserved from legacy ais.ts
- Hybrid fetch: proto RPC for snapshot-only, raw WS relay for candidates
- Proto-to-legacy type mapping for AisDisruptionEvent and AisDensityZone
- Exports fetchAisSignals, initAisStream, disconnectAisStream, getAisStatus, isAisConfigured, registerAisCallback, unregisterAisCallback, AisPositionData

* feat(2L-02): rewire consumer imports and delete 3 legacy maritime files

- cable-activity.ts: fetch NGA warnings via MaritimeServiceClient.listNavigationalWarnings() with NgaWarning shape reconstruction from proto fields
- military-vessels.ts: imports updated from './ais' to './maritime'
- Services barrel: updated from './ais' to './maritime'
- desktop-readiness.ts: service/api references updated to maritime handler paths
- Deleted: api/ais-snapshot.js, api/nga-warnings.js, src/services/ais.ts
- AisDisruptionEvent/AisDensityZone/AisDisruptionType preserved in src/types/index.ts

* docs(2L-02): complete maritime service module plan

- SUMMARY.md with hybrid fetch pattern, polling/callback preservation, 3 legacy files deleted
- STATE.md updated: phase 2L complete, 5 decisions recorded
- ROADMAP.md updated: 2L plans marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: bind globalThis.fetch in all sebuf service clients

Generated sebuf clients store globalThis.fetch as a class property,
then call it as this.fetchFn(). This loses the window binding and
throws "Illegal invocation" in browsers. Pass { fetch: fetch.bind(globalThis) }
to all 11 client constructors.

Also includes vite.config.ts with all 10 migrated domain handlers
registered in the sebuf dev server plugin.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate cyber + economic domains to sebuf (12/17)

Cyber (Phase 2M):
- Create handler aggregating 5 upstream sources (Feodo, URLhaus, C2Intel, OTX, AbuseIPDB)
  with dedup, GeoIP hydration, country centroid fallback
- Create service module with CyberServiceClient + circuit breaker
- Delete api/cyber-threats.js, api/cyber-threats.test.mjs, src/services/cyber-threats.ts

Economic (Phase 2N) — consolidates 3 legacy services:
- Create handler with 3 RPCs: getFredSeries (FRED API), listWorldBankIndicators
  (World Bank API), getEnergyPrices (EIA API)
- Create unified service module replacing fred.ts, oil-analytics.ts, worldbank.ts
- Preserve all exported functions/types for EconomicPanel and TechReadinessPanel
- Delete api/fred-data.js, api/worldbank.js, src/services/fred.ts,
  src/services/oil-analytics.ts, src/services/worldbank.ts

Both domains registered in vite.config.ts and api/[[...path]].ts.
TypeScript check and vite build pass cleanly.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate infrastructure domain to sebuf (13/17)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate market domain to sebuf (14/17)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate news domain to sebuf (15/17)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate intelligence domain to sebuf (16/17)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate military domain to sebuf (17/17)

All 17 domains now have sebuf handlers registered in the gateway.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate intelligence services to sebuf client

Rewire pizzint.ts, cached-risk-scores.ts, and threat-classifier.ts
to use IntelligenceServiceClient instead of legacy /api/ fetch calls.
Handler now preserves raw threat level in subcategory field.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate military theater posture to sebuf client

Rewire cached-theater-posture.ts to use MilitaryServiceClient instead
of legacy /api/theater-posture fetch. Adds theater metadata map for
proto→legacy TheaterPostureSummary adapter. UI gracefully falls back
to total counts when per-type breakdowns aren't available.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: rewire country-intel to sebuf client

Replace legacy fetch('/api/country-intel') with typed
IntelligenceServiceClient.getCountryIntelBrief() RPC call in App.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate stablecoin-markets to sebuf (market domain)

Add ListStablecoinMarkets RPC to market service. Port CoinGecko
stablecoin peg-health logic from api/stablecoin-markets.js into
the market handler. Rewire StablecoinPanel to use typed sebuf client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate etf-flows to sebuf (market domain)

Add ListEtfFlows RPC to market service. Port Yahoo Finance BTC spot
ETF flow estimation logic from api/etf-flows.js into the market handler.
Rewire ETFFlowsPanel to use typed sebuf client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate worldpop-exposure to sebuf (displacement domain)

Add GetPopulationExposure RPC to displacement service. Port country
population data and radius-based exposure estimation from
api/worldpop-exposure.js into the displacement handler. Rewire
population-exposure.ts to use typed sebuf client.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: remove superseded legacy edge functions

Delete 4 legacy api/*.js files that are now fully replaced by
sebuf handlers:
- api/stablecoin-markets.js -> market/ListStablecoinMarkets
- api/etf-flows.js -> market/ListEtfFlows
- api/worldpop-exposure.js -> displacement/GetPopulationExposure
- api/classify-batch.js -> intelligence/ClassifyEvent

Remaining legacy files are still actively used by client code
(stock-index, opensky, gdelt-doc, rss-proxy, summarize endpoints,
macro-signals, tech-events) or are shared utilities.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: delete dead legacy files and unused API_URLS config

Remove coingecko.js, debug-env.js, cache-telemetry.js, _cache-telemetry.js
(all zero active consumers). Delete unused API_URLS export from base config.
Update desktop-readiness market-panel metadata to reference sebuf paths.
Remove dead CoinGecko dev proxy from vite.config.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat: migrate stock-index and opensky to sebuf

- Add GetCountryStockIndex RPC to market domain (Yahoo Finance + cache)
- Fill ListMilitaryFlights stub in military handler (OpenSky with bounding box)
- Rewire App.ts stock-index fetch to MarketServiceClient.getCountryStockIndex()
- Delete api/stock-index.js and api/opensky.js edge functions
- OpenSky client path unchanged (relay primary, vite proxy for dev)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* wip: sebuf legacy migration paused at phase 3/10

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(03): capture phase context

* docs(state): record phase 3 context session

* docs(03): research phase domain

* docs(03): create phase plan — 5 plans in 2 waves

* feat(03-01): commit wingbits migration (step 3) -- 3 RPCs added to military domain

- Add GetAircraftDetails, GetAircraftDetailsBatch, GetWingbitsStatus RPCs
- Rewire src/services/wingbits.ts to use MilitaryServiceClient
- Update desktop-readiness.ts routes to match new RPC paths
- Delete legacy api/wingbits/ edge functions (3 files)
- Regenerate military service client/server TypeScript + OpenAPI docs

* feat(03-02): add SummarizeArticle proto and implement handler

- Create summarize_article.proto with request/response messages
- Add SummarizeArticle RPC to NewsService proto
- Implement full handler with provider dispatch (ollama/groq/openrouter)
- Port cache key builder, deduplication, prompt builder, think-token stripping
- Inline Upstash Redis helpers for edge-compatible caching

* feat(03-01): migrate gdelt-doc to intelligence RPC + delete _ip-rate-limit.js

- Add SearchGdeltDocuments RPC to IntelligenceService proto
- Implement searchGdeltDocuments handler (port from api/gdelt-doc.js)
- Rewire src/services/gdelt-intel.ts to use IntelligenceServiceClient
- Delete legacy api/gdelt-doc.js edge function
- Delete dead api/_ip-rate-limit.js (zero importers)
- Regenerate intelligence service client/server TypeScript + OpenAPI docs

* feat(03-02): rewire summarization client to NewsService RPC, delete 4 legacy files

- Replace direct fetch to /api/{provider}-summarize with NewsServiceClient.summarizeArticle()
- Preserve identical fallback chain: ollama -> groq -> openrouter -> browser T5
- Delete api/groq-summarize.js, api/ollama-summarize.js, api/openrouter-summarize.js
- Delete api/_summarize-handler.js and api/_summarize-handler.test.mjs
- Update desktop-readiness.ts to reference new sebuf route

* feat(03-03): rewire MacroSignalsPanel to EconomicServiceClient + delete legacy

- Replace fetch('/api/macro-signals') with EconomicServiceClient.getMacroSignals()
- Add mapProtoToData() to convert proto optional fields to null for rendering
- Delete legacy api/macro-signals.js edge function

* feat(03-04): add ListTechEvents proto, city-coords data, and handler

- Create list_tech_events.proto with TechEvent, TechEventCoords messages
- Add ListTechEvents RPC to ResearchService proto
- Extract 360-city geocoding table to api/data/city-coords.ts
- Implement listTechEvents handler with ICS+RSS parsing, curated events, dedup, filtering
- Regenerate TypeScript client/server from proto

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(03-01): complete wingbits + GDELT doc migration plan

- Create 03-01-SUMMARY.md with execution results
- Update STATE.md with plan 01 completion, steps 3-4 done
- Update ROADMAP.md plan progress (2/5 plans complete)
- Mark DOMAIN-10 requirement complete

* docs(03-02): complete summarization migration plan

- Create 03-02-SUMMARY.md with execution results
- Update STATE.md position to step 6/10
- Update ROADMAP.md plan progress
- Mark DOMAIN-09 requirement complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(03-04): rewire TechEventsPanel and App to ResearchServiceClient, delete legacy

- Replace fetch('/api/tech-events') with ResearchServiceClient.listTechEvents() in TechEventsPanel
- Replace fetch('/api/tech-events') with ResearchServiceClient.listTechEvents() in App.loadTechEvents()
- Delete legacy api/tech-events.js (737 lines)
- TypeScript compiles cleanly with no references to legacy endpoint

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(03-03): complete macro-signals migration plan

- Create 03-03-SUMMARY.md with execution results
- Mark DOMAIN-04 requirement complete in REQUIREMENTS.md

* docs(03-04): complete tech-events migration plan

- Add 03-04-SUMMARY.md with execution results
- Update STATE.md: advance to plan 5/step 8, add decisions
- Update ROADMAP.md: 4/5 plans complete for phase 03

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(03-05): add temporal baseline protos + handler with Welford's algorithm

- GetTemporalBaseline RPC: anomaly detection with z-score thresholds
- RecordBaselineSnapshot RPC: batch update via Welford's online algorithm
- Inline mgetJson helper for Redis batch reads
- Inline getCachedJson/setCachedJson Redis helpers
- Generated TypeScript client/server + OpenAPI docs

* feat(03-05): migrate temporal-baseline + tag non-JSON + final cleanup

- Rewire temporal-baseline.ts to InfrastructureServiceClient RPCs
- Delete api/temporal-baseline.js (migrated to sebuf handler)
- Delete api/_upstash-cache.js (no importers remain)
- Tag 6 non-JSON edge functions with // Non-sebuf: comment header
- Update desktop-readiness.ts: fix stale cloudflare-outages reference

* docs(03-05): complete temporal-baseline + non-JSON tagging + final cleanup plan

- SUMMARY.md with Welford algorithm migration details
- STATE.md updated: Phase 3 complete (100%)
- ROADMAP.md updated: 5/5 plans complete

* chore(03): delete orphaned ollama-summarize test after RPC migration

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-3): complete phase execution

* docs(v1): create milestone audit report

Audits all 13 phases of the v1 sebuf integration milestone.
12/13 phases verified (2L maritime missing VERIFICATION.md).
25/34 requirements satisfied, 6 superseded, 2 partial, 1 unsatisfied (CLEAN-03).
All 17 domains wired end-to-end. Integration check passes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(v1): update audit — mark CLEAN-03/04 + MIGRATE-* as superseded

CLEAN-03 superseded by port/adapter architecture (internal types
intentionally decoupled from proto wire types). MIGRATE-01-05
superseded by direct cutover approach. DOMAIN-03 checkbox updated.
Milestone status: tech_debt (no unsatisfied requirements).

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(roadmap): add gap closure phase 4 — v1 milestone cleanup

Closes all audit gaps: CLIENT-03 circuit breaker coverage, DOMAIN-03/06
verification gaps, documentation staleness, orphaned code cleanup.
Fixes traceability table phase assignments to match actual roadmap phases.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore(03): commit generated NewsService OpenAPI specs + checkpoint update

SummarizeArticle RPC was added during Phase 3 plan 02 but generated
OpenAPI specs were not staged with that commit.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(04): research phase domain

* docs(04): create phase plan

* docs(04-01): fix ROADMAP.md Phase 3 staleness and delete .continue-here.md

- Change Phase 3 heading from IN PROGRESS to COMPLETE
- Check off plans 03-03, 03-04, 03-05 (all were already complete)
- Delete stale .continue-here.md (showed task 3/10 in_progress but all 10 done)

* feat(04-02): add circuit breakers to seismology, wildfire, climate, maritime

- Seismology: wrap listEarthquakes in breaker.execute with empty-array fallback
- Wildfire: replace manual try/catch with breaker.execute for listFireDetections
- Climate: replace manual try/catch with breaker.execute for listClimateAnomalies
- Maritime: wrap proto getVesselSnapshot RPC in snapshotBreaker.execute, preserve raw relay fallback

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(04-02): add circuit breakers to news summarization and GDELT intelligence

- Summarization: wrap newsClient.summarizeArticle in summaryBreaker.execute for both tryApiProvider and translateText
- GDELT: wrap client.searchGdeltDocuments in gdeltBreaker.execute, replace manual try/catch
- Fix: include all required fields (tokens, reason, error, errorType, query) in fallback objects
- CLIENT-03 fully satisfied: all 17 domains have circuit breaker coverage

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(04-01): create 2L-VERIFICATION.md, fix desktop-readiness.ts, complete service barrel

- Create retroactive 2L-VERIFICATION.md with 12/12 must-haves verified
- Fix map-layers-core and market-panel stale file refs in desktop-readiness.ts
- Fix opensky-relay-cloud stale refs (api/opensky.js deleted)
- Add missing barrel re-exports: conflict, displacement, research, wildfires, climate
- Skip military/intelligence/news barrels (would cause duplicate exports)
- TypeScript compiles cleanly with zero errors

* docs(04-02): complete circuit breaker coverage plan

- SUMMARY.md: 6 domains covered, CLIENT-03 satisfied, 1 deviation (fallback type fix)
- STATE.md: Phase 4 plan 02 complete, position and decisions updated
- ROADMAP.md: Phase 04 marked complete (2/2 plans)
- REQUIREMENTS.md: CLIENT-03 marked complete

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(04-01): complete documentation fixes plan

- Create 04-01-SUMMARY.md with execution results
- Update STATE.md with Plan 01 completion and decisions
- Update ROADMAP.md: Plan 04-01 checked, progress 1/2

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs(phase-04): complete phase verification and fix tracking gaps

ROADMAP.md Phase 4 status updated to Complete, 04-02 checkbox checked,
progress table finalized. REQUIREMENTS.md coverage summary updated
(27 complete, 0 partial/pending). STATE.md reflects verified phase.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(military): migrate USNI fleet tracker to sebuf RPC

Port all USNI Fleet Tracker parsing logic from api/usni-fleet.js into
MilitaryService.GetUSNIFleetReport RPC with proto definitions, inline
Upstash caching (6h fresh / 7d stale), and client adapter mapping.
Deletes legacy edge function and _upstash-cache.js dependency.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* refactor: add proto validation annotations and split all 17 handler files into per-RPC modules

Phase A: Added buf.validate constraints to ~25 proto files (~130 field annotations
including required IDs, score ranges, coordinate bounds, page size limits).

Phase B: Split all 17 domain handler.ts files into per-RPC modules with thin
re-export handler.ts files. Extracted shared Redis cache helpers to
api/server/_shared/redis.ts.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: add ADDING_ENDPOINTS guide and update Contributing section

All JSON endpoints must use sebuf — document the complete workflow for
adding RPCs to existing services and creating new services, including
proto conventions, validation annotations, and generated OpenAPI docs.

Update DOCUMENTATION.md Contributing section to reference the new guide
and remove the deprecated "Adding a New API Proxy" pattern.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add blank lines before lists to pass markdown lint (MD032)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: disambiguate duplicate city keys in city-coords

Vercel's TypeScript check treats duplicate object keys as errors (TS1117).
Rename 'san jose' (Costa Rica) -> 'san jose cr' and
'cambridge' (UK) -> 'cambridge uk' to avoid collision.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: resolve Vercel deployment errors — relocate hex-db + fix TS strict-null

- Move military-hex-db.js next to handler (fixes Edge Function unsupported module)
- Fix strict-null TS errors across 12 handler files (displacement, economic,
  infrastructure, intelligence, market, military, research, wildfire)
- Add process declare to wildfire handler, prefix unused vars, cast types

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: convert military-hex-db to .ts for Edge Function compatibility

Vercel Edge bundler can't resolve .js data modules from .ts handlers.
Also remove unused _region variable.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: inline military hex db as packed string to avoid Edge Function module error

Vercel Edge bundler can't resolve separate data modules. Inline 20K hex
IDs as a single concatenated string, split into Set at runtime.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove broken export { process } from 4 _shared files

declare const is stripped in JS output, making export { process }
reference nothing. No consumers import it — each handler file has
its own declare.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: move server/ out of api/ to fix Vercel catch-all routing

Vercel's file-based routing was treating api/server/**/*.ts as individual
API routes, overriding the api/[[...path]].ts catch-all for multi-segment
paths like /api/infrastructure/v1/list-service-statuses (3 segments).
Moving to server/ at repo root removes the ambiguity.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: rename catch-all to [...path] — [[...path]] is Next.js-only syntax

Vercel's native edge function routing only supports [...path] for
multi-segment catch-all matching. The [[...path]] double-bracket syntax
is a Next.js feature and was only matching single-segment paths.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add dynamic segment route for multi-segment API paths

Vercel's native file-based routing for non-Next.js projects doesn't
support [...path] catch-all matching multiple segments. Use explicit
api/[domain]/v1/[rpc].ts which matches /api/{domain}/v1/{rpc} via
standard single-segment dynamic routing that Vercel fully supports.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: remove conflicting [...path] catch-all — replaced by [domain]/v1/[rpc]

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: widen CORS pattern to match all Vercel preview URL formats

Preview URLs use elie-ab2dce63 not elie-habib-projects as the team slug.
Broaden pattern to elie-[a-z0-9]+ to cover both.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: update sidecar build script for new api/[domain]/v1/[rpc] path

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: trigger Vercel rebuild for all variants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 review — critical bugs, hardening, and cleanup

Fixes from @koala73's code review:

Critical:
- C-1: Add max-size eviction (2048 cap) to GeoIP in-memory cache
- C-2: Move type/source/severity filters BEFORE .slice(pageSize) in cyber handler
- C-3: Atomic SET with EX in Redis helper (single Upstash REST call)
- C-4: Add AbortSignal.timeout(30s) to LLM fetch in summarize-article

High:
- H-1: Add top-level try/catch in gateway with CORS-aware 500 response
- H-3: Sanitize error messages — generic text for 5xx, passthrough for 4xx only
- H-4: Add timeout (10s) + Redis cache (5min) to seismology handler
- H-5: Add null guards (optional chaining) in seismology USGS feature mapping
- H-6: Race OpenSky + Wingbits with Promise.allSettled instead of sequential fallback
- H-8: Add Redis cache (5min TTL) to infrastructure service-status handler

Medium:
- M-12/M-13: Fix HAPI summary field mappings (iso3 from countryCode, internallyDisplaced)

Infrastructure:
- R-1: Remove .planning/ from git tracking, add to .gitignore
- Port UCDP parallel page fetching from main branch (#198)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 review issues — hash, CORS, router, cache, LLM, GeoIP

Fixes/improvements from PR #106 code review tracking issues:

- #180: Replace 32-bit hash (Java hashCode/DJB2) with unified FNV-1a 52-bit
  hash in server/_shared/hash.ts, greatly reducing collision probability
- #182: Cache router construction in Vite dev plugin — build once, invalidate
  on HMR changes to server/ files
- #194: Add input length limits for LLM prompt injection (headlines 500 chars,
  title 500 chars, geoContext 2000 chars, max 10 headlines)
- #195/#196: GeoIP AbortController — cancel orphaned background workers on
  timeout instead of letting them fire after response is sent
- #198: Port UCDP partial-result caching from main — 10min TTL for partial
  results vs 6hr for complete, with in-memory fallback cache

Proto codegen regenerated for displacement + conflict int64_encoding changes.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: restore fast-xml-parser dependency needed by sebuf handlers

Main branch removed fast-xml-parser in v2.5.1 (legacy edge functions
no longer needed it), but sebuf handlers in aviation/_shared.ts and
research/list-arxiv-papers.ts still import it for XML API parsing.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: fix stale paths, version badge, and local-backend-audit for sebuf

- ADDING_ENDPOINTS.md: fix handler paths from api/server/ to server/,
  fix import depth (4 levels not 5), fix gateway extension (.js not .ts)
- DOCUMENTATION.md: update version badge 2.1.4 -> 2.5.1, fix broken
  ROADMAP.md links to .planning/ROADMAP.md, fix handler path reference
- COMMUNITY-PROMOTION-GUIDE.md: add missing v2.5.1 to version table
- local-backend-audit.md: rewrite for sebuf architecture — replace all
  stale api/*.js references with sebuf domain handler paths

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* docs: use make commands, add generation-before-push warning, bump sebuf to v0.7.0

- ADDING_ENDPOINTS.md: replace raw `cd proto && buf ...` with `make check`,
  `make generate`, `make install`; add warning that `make generate` must run
  before pushing proto changes (links to #200)
- Makefile: bump sebuf plugin versions from v0.6.0 to v0.7.0
- PR description also updated to use make commands

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* chore: make install installs everything, document setup

- Makefile: `make install` now installs buf, sebuf plugins, npm deps,
  and proto deps in one command; pin buf and sebuf versions as variables
- ADDING_ENDPOINTS.md: updated prerequisites to show `make install`
- DOCUMENTATION.md: updated Installation section with `make install`
  and generation reminder

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 re-review issues — timeouts, iso3, pagination, CORS, dedup

- NEW-1: HAPI handler now returns ISO-3 code (via ISO2_TO_ISO3 lookup) instead of ISO-2
- NEW-3: Aviation FAA fetch now has AbortSignal.timeout(15s)
- NEW-4: Climate Open-Meteo fetch now has AbortSignal.timeout(20s)
- NEW-5: Wildfire FIRMS fetch now has AbortSignal.timeout(15s)
- NEW-6: Seismology now respects pagination.pageSize (default 500)
- NEW-9: Gateway wraps getCorsHeaders() in try/catch with safe fallback
- NEW-10: Tech events dedup key now includes start year to avoid dropping yearly variants

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 round 2 — seismology crash, tsconfig, type contracts

- Fix _req undefined crash in seismology handler (renamed to req)
- Cache full earthquake set, slice on read (avoids cache pollution)
- Add server/ to tsconfig.api.json includes (catches type errors at build)
- Remove String() wrappers on numeric proto fields in displacement/HAPI
- Fix hashString re-export not available locally in news/_shared.ts

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: loadMarkets heatmap regression, stale test, Makefile playwright

- Add finnhub_skipped + skip_reason to ListMarketQuotesResponse proto
- Wire skipped signal through handler → adapter → App.loadMarkets
- Fix circuit breaker cache conflating different symbol queries (cacheTtlMs: 0)
- Use dynamic fetch wrapper so e2e test mocks intercept correctly
- Update e2e test mocks from old endpoints to sebuf proto endpoints
- Delete stale summarization-chain.test.mjs (imports deleted files)
- Add install-playwright target to Makefile

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: add missing proto fields to emptyStockFallback (build fix)

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(quick-1): fix country fallback, ISO-2 contract, and proto field semantics

- BLOCKING-1: Return undefined when country has no ISO3 mapping instead of wrong country data
- BLOCKING-2: country_code field now returns ISO-2 per proto contract
- MEDIUM-1: Rename proto fields from humanitarian to conflict-event semantics (populationAffected -> conflictEventsTotal, etc.)
- Update client service adapter to use new field names
- Regenerate TypeScript types from updated proto

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* feat(quick-1): add in-memory cache + in-flight dedup to AIS vessel snapshot

- HIGH-1: 10-second TTL cache matching client poll interval
- Concurrent requests share single upstream fetch (in-flight dedup)
- Follows same pattern as get-macro-signals.ts cache
- No change to RPC response shape

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(quick-1): stub RPCs throw UNIMPLEMENTED, remove hardcoded politics, add tests

- HIGH-2: listNewsItems, summarizeHeadlines, listMilitaryVessels now throw UNIMPLEMENTED
- LOW-1: Replace hardcoded "Donald Trump" with date-based dynamic LLM context
- LOW-1 extended: Also fix same issue in intelligence/get-country-intel-brief.ts (Rule 2)
- MEDIUM-2: Add tests/server-handlers.test.mjs with 20 tests covering all review items

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix(quick-2): remove 3 dead stub RPCs (ListNewsItems, SummarizeHeadlines, ListMilitaryVessels)

- Delete proto definitions, handler stubs, and generated code for dead RPCs
- Clean _shared.ts: remove tryGroq, tryOpenRouter, buildPrompt, dead constants
- Remove 3 UNIMPLEMENTED stub tests from server-handlers.test.mjs
- Regenerate proto codegen (buf generate) and OpenAPI docs
- SummarizeArticle and all other RPCs remain intact

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* fix: address PR #106 review findings (stale snapshot, iso naming, scoring, test import)

- Serve stale AIS snapshot on relay failure instead of returning undefined
- Rename HapiConflictSummary.iso3 → iso2 to match actual ISO-2 content
- Fix HAPI fallback scoring: use weight 3 for combined political violence
  (civilian targeting is folded in, was being underweighted at 0)
- Extract deduplicateHeadlines to shared .mjs so tests import production code

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>

* test: add coverage for PR106 iso2 and fallback regressions

---------

Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com>
Co-authored-by: Elie Habib <elie.habib@gmail.com>
 2026-02-21 03:39:56 +04:00