149 Commits

Author SHA1 Message Date
Elie Habib
3e9556c37f feat(resilience): Phase 1 §3.2+§3.3 — full-country sanctions counts + grey-out ranking (#2763)
* feat(resilience): Phase 1 §3.2+§3.3 — full-country sanctions counts + grey-out ranking

§3.2 — Switch sanctions from top-12 to full country counts
- RESILIENCE_SANCTIONS_KEY: 'sanctions:pressure:v1' → 'sanctions:country-counts:v1'
- New key is a plain ISO2→entryCount map covering ALL countries (no top-12 truncation)
- Replaces compound pressure formula with normalizeSanctionCount() piecewise scale:
  0=100, 1-10=90-75, 11-50=75-50, 51-200=50-25, 201+=25→0
- IMPUTE.ofacSanctions removed (country-counts covers all countries; no absent-country
  imputation needed for sanctions)

§3.3 — Grey-out criteria for ranking
- Proto: ResilienceRankingItem.overall_coverage (field 5) + GetResilienceRankingResponse.greyed_out
- GREY_OUT_COVERAGE_THRESHOLD = 0.40: countries below this are excluded from ranking
  but still appear on choropleth in "insufficient data" style
- buildRankingItem() now computes overallCoverage from domain/dimension data
- getResilienceRanking() splits items into ranked (≥0.40) + greyedOut (<0.40)

Tests updated for new sanctions format; overall score anchor updated (67.56).

* fix(resilience): fix ranking cache guard for all-greyed-out + stale shape cases

Two cache bugs:

1. Empty-items guard: `cached?.items?.length` fails when every country falls
   below GREY_OUT_COVERAGE_THRESHOLD (items=[], greyedOut=[…]). The cache was
   written correctly but never served, causing unnecessary rewarming on every
   request for sparse-data deployments.
   Fix: `cached != null && (items.length > 0 || greyedOut.length > 0)`

2. Stale-shape test: agent-written cache test stored a payload without
   `greyedOut` or `overallCoverage`, locking in pre-PR shape. Updated to the
   correct post-deploy shape so the test reflects actual cached content.

Cache key was already bumped to resilience:ranking:v2 (forces fresh compute
on first post-deploy request, avoiding old-shape responses in production).

* fix(resilience): consume greyedOut on choropleth; version ranking cache key

- Add 'insufficient_data' level to ResilienceChoroplethLevel and RESILIENCE_CHOROPLETH_COLORS
- Extend buildResilienceChoroplethMap to accept optional greyedOut array
- Thread greyedOut through DeckGLMap.setResilienceRanking, MapContainer.setResilienceRanking (with replay), and data-loader.ts
- Add 'Insufficient data' tooltip guard for greyed-out countries in DeckGLMap
- Bump RESILIENCE_RANKING_CACHE_KEY to resilience:ranking:v2 to invalidate stale schema-mismatched cache entries
- Update api/health.js probe key to match

* fix(resilience): include greyedOut in seed-meta count to avoid false health alert

seed-meta:resilience:ranking was written with count=response.items.length,
which excludes greyedOut countries. In an all-greyed-out deployment, count=0
causes api/health.js to report the ranking as EMPTY_DATA/critical even though
the cached payload is valid (items:[], greyedOut:[…]).

Fix: count = items.length + greyedOut.length — total scoured countries
regardless of ranking eligibility.

* test(resilience): pin all-greyed-out cache-hit regression

Adds the missing test case: cached payload with items=[] and greyedOut=[…]
must be served from cache without triggering score rewarming.

Previously, `cached?.items?.length` was falsy for this shape, making the
guard ineffective. The fix (items.length > 0 || greyedOut.length > 0) was
correct but unpinned — this test locks it in.
2026-04-06 14:19:16 +04:00
Elie Habib
5dbc72d7c6 feat(energy): GetCountryEnergyProfile RPC — aggregate Phase 1/2/2.5 data per country (#2747)
* feat(energy): GetCountryEnergyProfile RPC — aggregate Phase 1/2/2.5 data per country

Add new intelligence RPC that reads 6 Redis keys in parallel (OWID mix,
EU gas storage, electricity prices, JODI oil, JODI gas, IEA oil stocks)
and returns a unified energy profile per ISO2 country code. All signals
are optional with graceful omission on missing/null keys. Also register
the new route in gateway.ts with slow cache tier.

* fix(energy): convert gasLngShare from 0-1 fraction to 0-100 percentage to match all other share fields

* fix(energy): add lpgImportsKbd, fix electricityAvailable source/date consistency, drop false US electricity claim

* chore(proto): regenerate OpenAPI specs after energy profile field description update
2026-04-05 23:50:31 +04:00
Fayez Bast
8609ad1384 feat: " climate disasters alerts seeders " (#2550)
* Revert "Revert "feat(climate): add climate disasters seed + ListClimateDisast…"

This reverts commit ae4010a795.

* feat(climate):add-disaster-alerts-seeder

* fix(climate): review fixes for climate disasters seeder

- Bump CACHE_TTL from 6h to 18h (gold standard: TTL >= 3x cron interval)
- Log warning when ReliefWeb rows all map to null (aids debugging schema changes)
- Anchor getNaturalSourceMeta to known source names/URLs (prevents false positives)
- Normalize seed output to camelCase (matches proto field names, simplifies handler)

* fix(climate): fail hard on config errors, drop Null Island records

- Config errors (missing RELIEFWEB_APPNAME) now propagate through
  collectDisasterSourceResults instead of being tolerated as partial
  failures. Transient errors (e.g. natural cache unavailable) are
  still tolerated.
- Drop ReliefWeb and natural-event records with no resolvable country
  code instead of emitting (0,0) Null Island points.
- Add test for config error hard-fail behavior.

* fix(climate): tag rejected appname as config error for hard-fail

fetchReliefWeb now tags HTTP 401/403 responses as isConfigError,
so collectDisasterSourceResults fails the entire seed instead of
tolerating it as a partial failure. Covers both missing and
invalid/unapproved RELIEFWEB_APPNAME cases.

* chore: regenerate OpenAPI spec after merge

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-04-04 23:18:53 +04:00
Elie Habib
4e9f25631c feat(economic): add FAO Food Price Index panel (#2682)
* feat(economic): add FAO Food Price Index panel

Adds a new panel tracking the FAO Global Food Price Index (FFPI) for the
past 12 months, complementing existing consumer prices, fuel prices, and
Big Mac Index trackers.

- proto: GetFaoFoodPriceIndex RPC with 6-series response (Food, Cereals,
  Meat, Dairy, Oils, Sugar + MoM/YoY pct)
- seeder: seed-fao-food-price-index.mjs with 90-day TTL (3× monthly),
  isMain guard, parseVal NaN safety, correct 13-point slice
- handler/gateway: static tier RPC wired into economicHandler
- bootstrap/health: bootstrapped as SLOW_KEY; maxStaleMin=86400 (60 days)
- panel: SVG multi-line chart with 6 series, auto-scaled Y axis, headline
  with MoM/YoY indicators, info tooltip, bootstrap hydration
- CMD+K: panel:fao-food-price-index with fao/ffpi/food keywords
- Railway: fao-ffpi cron seeder service (0.5 vCPU, 0.5 GB, daily 08:45)
- locales: full en.json keys for panel UI strings
- ais-relay: faoFoodPriceIndex added to economic bootstrap context

* fix(economic): add faoFoodPriceIndex to cache-keys.ts BOOTSTRAP_CACHE_KEYS and BOOTSTRAP_TIERS

* fix(economic): correct cron comment in fao seeder to reflect daily schedule
2026-04-04 17:33:54 +04:00
Fayez Bast
02f55dc584 feat(climate): add ocean ice indicators seed and RPC (#2652)
* feat(climate): add ocean ice indicators seed and RPC

* fix(review): restore MCP maxStaleMin, widen health threshold, harden sea level parser, type globe.gl shim

- Restore get_climate_data _maxStaleMin to 2880 (was accidentally lowered to 1440)
- Bump oceanIce SEED_META maxStaleMin from 1440 to 2880 (2× daily interval, tolerates one missed run)
- Add fallback regex patterns for NASA sea level overlay HTML parsing
- Replace globe.gl GlobeInstance `any` with typed interface (index sig stays `any` for Three.js compat)

* fix(review): merge prior cache on partial failures, fix fallback regex, omit trend without baseline

- P1: fetchOceanIceData() now reads prior cache and merges last-known-good
  indicators when any upstream source fails, preventing partial overwrites
  from erasing previously healthy data
- P1: sea level fallback regex now requires "current" context to avoid
  matching the historical 1993 baseline rate instead of the current rate
- P2: classifyArcticTrend() returns null (omitted from payload) when no
  climatology baseline exists, instead of misleadingly labeling as "average"
- Added tests for all three fixes

* fix(review): merge prior cache by source field group, not whole object

Prior-cache merge was too coarse: Object.assign(payload, priorCache)
reintroduced stale arctic_extent_anomaly_mkm2 and arctic_trend from
prior cache when sea-ice succeeded but intentionally omitted those
fields (no climatology baseline), and an unrelated source like OHC
or sea level failed in the same run.

Fix: define per-source field groups (seaIce, seaLevel, ohc, sst).
Only fall back to prior cache fields for groups whose source failed
entirely. When a source succeeds, only its returned fields appear
in the payload, even if it omits fields it previously provided.

Added test covering the exact combined case: sea-ice climatology
unavailable + unrelated source failure + prior-cache merge enabled.

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-04-04 08:11:49 +04:00
Lucas Passos
4b67012260 feat(resilience): add service proto and stub handlers (#2657)
* feat(resilience): add service proto and stub handlers

Add the worldmonitor.resilience.v1 proto package, generated client/server artifacts, edge routing, and zero-state handler stubs so the domain is deployable before the seed and scoring layers land.

Validation:
- PATH="/Users/lucaspassos/go/bin:/Users/lucaspassos/.codex/tmp/arg0/codex-arg06nbVvG:/Users/lucaspassos/.antigravity/antigravity/bin:/Users/lucaspassos/.local/bin:/Users/lucaspassos/.codeium/windsurf/bin:/Users/lucaspassos/Library/Python/3.12/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/pkg/env/active/bin:/opt/pmk/env/global/bin:/Library/Apple/usr/bin:/opt/homebrew/bin:/Applications/Codex.app/Contents/Resources" make generate
- PATH="/Users/lucaspassos/go/bin:/Users/lucaspassos/.codex/tmp/arg0/codex-arg06nbVvG:/Users/lucaspassos/.antigravity/antigravity/bin:/Users/lucaspassos/.local/bin:/Users/lucaspassos/.codeium/windsurf/bin:/Users/lucaspassos/Library/Python/3.12/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/pkg/env/active/bin:/opt/pmk/env/global/bin:/Library/Apple/usr/bin:/opt/homebrew/bin:/Applications/Codex.app/Contents/Resources" npx tsx --test tests/route-cache-tier.test.mjs tests/edge-functions.test.mjs
- npm run typecheck (fails on upstream Dodo/Clerk baseline)
- npm run typecheck:api (fails on upstream vitest baseline)
- npm run test:data (fails on upstream dodopayments-checkout baseline via tests/runtime-config-panel-visibility.test.mjs)

* fix(resilience): add countryCode validation to get-resilience-score

Throw ValidationError when countryCode is missing instead of silently
returning a zero-state response with an empty string country code.

* fix(resilience): validate countryCode format and mark required in spec

- Trim whitespace and reject non-ISO-3166-1 alpha-2 codes to prevent
  cache pollution from malformed aliases (e.g. 'USA', '  us  ', 'foobar')
- Add required: true to proto QueryConfig so generated OpenAPI spec
  matches runtime validation behavior
- Regenerated OpenAPI artifacts via make generate

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-04-04 08:04:46 +04:00
Fayez Bast
9d94ad36aa feat(climate+health):add shared air quality seed and mirrored health (#2634)
* feat(climate+health):add shared air quality seed and mirrored health/climate RPCs

* feat(climate+health):add shared air quality seed and mirrored health/climate RPCs

* fix(air-quality): address review findings — TTL, seed-health, FAST_KEYS, shared meta

- Raise CACHE_TTL from 3600 to 10800 (3× the 1h cron cadence; gold standard)
- Add health:air-quality to api/seed-health.js SEED_DOMAINS so monitoring dashboard tracks freshness
- Remove climateAirQuality and healthAirQuality from FAST_KEYS (large station payloads; load in slow batch)
- Point climateAirQuality SEED_META to same meta key as healthAirQuality (same seeder run, one source of truth)

* fix(bootstrap): move air quality keys to SLOW tier — large station payloads avoid critical-path batch

* fix(air-quality): fix malformed OpenAQ URL and remove from bootstrap until panel exists

- Drop deprecated first URL attempt (parameters=pm25, order_by=lastUpdated, sort=desc);
  use correct v3 params (parameters_id=2, sort_order=desc) directly — eliminates
  guaranteed 4xx retry cycle per page on 20-page crawl
- Remove climateAirQuality and healthAirQuality from BOOTSTRAP_CACHE_KEYS, SLOW_KEYS,
  and BOOTSTRAP_TIERS — no panel consumes these yet; adding thousands of station records
  to every startup bootstrap is pure payload bloat
- Remove normalizeAirQualityPayload helpers from bootstrap.js (no longer called)
- Update service wrappers to fetch via RPC directly; re-add bootstrap hydration
  when a panel actually needs it

* fix(air-quality): raise lock TTL to 3600s to cover 20-page crawl worst case

2 OpenAQ calls × 20 pages × (30s timeout × 3 attempts) = 3600s max runtime.
Previous 600s TTL allowed concurrent cron runs on any degraded upstream.

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-04-03 10:27:37 +04:00
Elie Habib
6c017998d3 feat(e3): story persistence tracking (#2620)
* feat(e3): story persistence tracking

Adds cross-cycle story tracking layer to the RSS digest pipeline:

- Proto: StoryMeta message + StoryPhase enum on NewsItem (fields 9-11).
  importanceScore and corroborationCount stubs added for E1.
- list-feed-digest.ts: builds corroboration map across ALL items before
  truncation; batch-reads existing story:track hashes from Redis; writes
  HINCRBY/HSET/HSETNX/SADD/EXPIRE per story in 80-story pipeline chunks;
  attaches StoryMeta (firstSeen, mentionCount, sourceCount, phase) to
  each proto item using read-back data.
- cache-keys.ts: STORY_TRACK_KEY_PREFIX, STORY_SOURCES_KEY_PREFIX,
  DIGEST_ACCUMULATOR_KEY_PREFIX, STORY_TRACKING_TTL_S.
- src/types/index.ts: StoryMeta, StoryPhase, NewsItem extended.
- data-loader.ts: protoItemToNewsItem maps STORY_PHASE_* → client phase.
- NewsPanel.ts: BREAKING/DEVELOPING/ONGOING phase badges in item rows.

New story first appearance: phase=BREAKING. After 2 mentions within 2h:
DEVELOPING. After 6+ mentions or >2h: SUSTAINED. If score drops below
50% of peak: FADING (used by E1; defaults to SUSTAINED for now).

Redis keys per story (48h TTL):
  story:track:v1:<hash16>   → hash (firstSeen,lastSeen,mentionCount,...)
  story:sources:v1:<hash16> → set  (feed names, for cross-source count)

* fix(e3): correct storyMeta staleness and mentionCount semantics

P1 — storyMeta was always one cycle behind because storyTracks was read
before writeStoryTracking ran. Fix: keep read-before-write but compute
storyMeta from merged in-memory state (stale.mentionCount + 1, fresh
sourceCount from corroborationMap). New stories get mentionCount=1 and
phase=BREAKING in the same cycle they first appear — no extra Redis
round-trip needed.

P2 — mentionCount incremented once per item occurrence, so a story seen
in 3 sources in its first cycle was immediately stored as mentionCount=3.
Fix: deduplicate by titleHash in writeStoryTracking so each unique story
gets exactly one HINCRBY per digest cycle regardless of source count.
SADD still collects all sources for the set key.

* fix(e3): Unicode hash collision, ALERT badge regression, FADING comment

P1 — normalizeTitle used [^\w\s] without the u flag; \w is ASCII-only
so every Arabic/CJK/Cyrillic title stripped to "" and shared one Redis
hash. Fixed: use /[^\p{L}\p{N}\s]/gu (Unicode property escapes require
the u flag).

P1 — ALERT badge was gated on !item.storyMeta, suppressing the indicator
for any tracked story regardless of isAlert. Phase and alert are
orthogonal signals; ALERT now renders unconditionally when isAlert=true.

P2 — FADING branch is intentionally inactive until E1 ships real scores
(currentScore/peakScore placeholder 0 via HSETNX). Added comment to
document the intentional ordering.

* fix(news-alerts): skip sustained/fading stories in breaking alert selectBest

Sustained and fading story phases are already well-covered by the feed;
only breaking and developing phases warrant a banner interrupt. Items
without storyMeta (phase unspecified) pass through unchanged.

Fixes gap C from docs/plans/2026-04-02-003-fix-news-alerts-pr-gaps-plan.md

* fix(e3): remove rebase artifacts from list-feed-digest

Removes a stray closing brace, duplicate ASCII normalizeTitle
(Unicode-aware version from the fix commit is correct), and
a leftover storyPhase assignment that references a removed field.

All typecheck and typecheck:api pass clean.
2026-04-02 21:16:35 +04:00
Elie Habib
8d8cf56ce2 feat(scoring): composite importance score + story tracking infrastructure (#2604)
* feat(scoring): add composite importance score + story tracking infrastructure

- Extract SOURCE_TIERS/getSourceTier to server/_shared/source-tiers.ts so
  server handlers can import it without pulling in client-only modules;
  src/config/feeds.ts re-exports for backward compatibility
- Add story tracking Redis key helpers to cache-keys.ts
  (story:track:v1, story:sources:v1, story:peak:v1, digest:accumulator:v1)
- Export SEVERITY_SCORES from _classifier.ts for server-side score math
- Add upstashPipeline() to redis.ts for arbitrary batched Redis writes
- Add importanceScore/corroborationCount/storyPhase fields to proto,
  generated TS, and src/types NewsItem
- Add StoryMeta message and StoryPhase enum to proto
- In list-feed-digest.ts:
  - Build corroboration map across full corpus BEFORE per-category truncation
  - Compute importanceScore (severity 40% + tier 20% + corroboration 30%
    + recency 10%) per item
  - Sort by importanceScore desc before truncating at MAX_ITEMS_PER_CATEGORY
  - Write story:track / story:sources / story:peak / digest:accumulator
    to Redis in 80-story pipeline batches after each digest build

Score gate in notification-relay.cjs follows in the next PR (shadow mode,
behind IMPORTANCE_SCORE_LIVE flag). RELAY_GATES_READY removal of
/api/notify comes after 48h shadow comparison confirms parity.

* fix(scoring): add storyPhase field + regenerate proto types

- Add storyPhase to ParsedItem and toProtoItem (defaults UNSPECIFIED)
- Regenerate service_server.ts: required fields, StoryPhase type relocated
- Regenerate service_client.ts and OpenAPI docs from buf generate
- Fix typecheck:api error on missing required storyPhase in NewsItem

* fix(scoring): address all code review findings from PR #2604

P1:
- await writeStoryTracking instead of fire-and-forget to prevent
  silent data loss on edge isolate teardown
- remove duplicate upstashPipeline; use existing runRedisPipeline
- strip non-https links before Redis write (XSS prevention)
- implement storyPhase read path: HGETALL batch + computePhase()
  so BREAKING/DEVELOPING/SUSTAINED/FADING badges are now live

P2/P3:
- extend STORY_TTL 48h → 7 days (sustained stories no longer reset)
- extract SCORE_WEIGHTS named constants with rationale comment
- move SEVERITY_SCORES out of _classifier.ts into list-feed-digest.ts
- add normalizeTitle comment referencing todo #102
- pre-compute title hashes once, share between phase read + write

* fix(scoring): correct enrichment-before-scoring and write-before-read ordering

Two sequencing bugs:

1. enrichWithAiCache ran after truncation (post-slice), so items whose
   threat level was upgraded by the LLM cache could have already been
   cut from the top-20, and downgraded items kept inflated scores.
   Fix: enrich ALL items from the full corpus before scoring, so
   importanceScore always uses the final post-LLM classification level.

2. Phase HGETALL read happened before writeStoryTracking, meaning
   first-time stories had no Redis entry and always returned UNSPECIFIED
   instead of BREAKING, and all existing stories lagged one cycle behind.
   Fix: write tracking first, then read back for phase assignment.
2026-04-02 20:46:04 +04:00
Fayez Bast
b2bae30bd8 Add climate news seed and ListClimateNews RPC (#2532)
* Add climate news seed and ListClimateNews RPC

* Wire climate news into bootstrap and fix generated climate stubs

* fix(climate): align seed health interval and parse Atom entries per feed

* fix(climate-news): TTL 90min, retry timer on failure, named cache key constant

- CACHE_TTL: 1800 to 5400 (90min = 3x 30-min relay interval, gold standard)
- ais-relay: add 20-min retry timer on subprocess failure; clear on success
- cache-keys.ts: export CLIMATE_NEWS_KEY named constant
- list-climate-news.ts: import CLIMATE_NEWS_KEY instead of hard-coding string

---------

Co-authored-by: Elie Habib <elie.habib@gmail.com>
2026-04-02 08:55:22 +04:00
Fayez Bast
bb4f8dcb12 feat(climate): add WMO normals seeding and CO2 monitoring (#2531)
* feat(climate): add WMO normals seeding and CO2 monitoring

* fix(climate): skip missing normals per-zone and align anomaly tooltip copy

* fix(climate): remove normals from bootstrap and harden health/cache key wiring

* feat(climate): version anomaly cache to v2, harden seed freshness, and align CO2/normal baselines
2026-04-02 08:17:32 +04:00
Elie Habib
ae4010a795 Revert "feat(climate): add climate disasters seed + ListClimateDisasters RPC with snake_case Redis payload (#2535)" (#2544)
This reverts commit e2dea9440d.
2026-03-30 13:09:19 +04:00
Fayez Bast
e2dea9440d feat(climate): add climate disasters seed + ListClimateDisasters RPC with snake_case Redis payload (#2535) 2026-03-30 12:23:32 +04:00
Elie Habib
dca83bb5e5 feat(forecast): simulation confidence sub-bar in ForecastPanel (#2526)
Add three fields to Forecast proto (simulation_adjustment,
sim_path_confidence, demoted_by_simulation) and implement a
thin colored underbar below each forecast title that encodes
simulation evidence without adding columns or text clutter.

Visual design (Option D):
- 2px colored bar, width = sim path confidence for positive adj,
  100% for negative/demoted (structural, not confidence-dependent)
- Green ≥0.70 conf, amber <0.70, orange negative, red demoted
- Opacity 0.45 at rest; 0.9 + text label on hover
- Plain language hover labels: "AI signal · +8%", "AI caution · −12%",
  "AI flag: dropped · −15%" — no "sim" jargon visible to users
- Demoted rows dim to opacity 0.5

Passes through simulation fields in buildPublishedForecastPayload.
No chip renders until the ExpandedPath → Forecast plumbing lands
(follow-up PR); all rendering code is ready and typecheck clean.

🤖 Generated with Claude Sonnet 4.6 via Claude Code + Compound Engineering v2.49.0

Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
2026-03-29 23:03:03 +04:00
Elie Habib
8aee4d340e feat(intelligence): GetCountryRisk RPC + MCP tool for per-country risk scores (#2502)
* feat(intelligence): GetCountryRisk RPC for per-country risk intelligence

Adds a new fast Redis-read RPC that consolidates CII score, travel advisory
level, and OFAC sanctions exposure into a single per-country response.

Replaces the need to call GetRiskScores (all-countries) and filter client-side.
Wired to MCP as get_country_risk tool (no LLM, ~200ms, good for agent screening).

- proto/intelligence/v1/get_country_risk.proto (new)
- server/intelligence/v1/get-country-risk.ts (reads 3 pre-seeded Redis keys)
- gateway.ts: slow cache tier
- api/mcp.ts: RpcToolDef with 8s timeout
- tests/mcp.test.mjs: update tool count 27→28

* fix(intelligence): upstream-unavailable signal, fetchedAt from CII, drop redundant catch

P1: return upstreamUnavailable:true when all Redis reads are null — prevents
CDN from caching false-negative sanctions/risk responses during Redis outages.

P2: fetchedAt now uses cii.computedAt (actual data age) instead of request time.

P2: removed redundant .catch(() => null) — getCachedJson already swallows errors.

* fix(intelligence): accurate OFAC counts and country names for GetCountryRisk

P1: sanctions:pressure:v1.countries is a top-12 slice — switch to a new
sanctions:country-counts:v1 key (ISO2→count across ALL 40K+ OFAC entries).
Written by seed-sanctions-pressure.mjs in afterPublish alongside entity index.

P1: trigger upstreamUnavailable:true when sanctions key alone is missing,
preventing false-negative sanctionsActive:false from being cached by CDN.

P2: advisory seeder now writes byCountryName (ISO2→display name) derived
from country-names.json reverse map. Handler uses it as fallback so countries
outside TIER1_COUNTRIES (TH, CO, BD, IT...) get proper names.
2026-03-29 17:07:03 +04:00
Elie Habib
aa3e84f0ab feat(economic): Economic Stress Composite Index panel (FRED 6-series, 0-100 score) (#2461)
* feat(economic): Economic Stress Composite Index panel (FRED 6-series, 0-100 score)

- Add T10Y3M and STLFSI4 to FRED_SERIES in seed-economy.mjs and ALLOWED_SERIES
- Create proto message GetEconomicStressResponse with EconomicStressComponent
- Register GetEconomicStress RPC in EconomicService (GET /api/economic/v1/get-economic-stress)
- Add seed-economic-stress.mjs: reads 6 pre-seeded FRED keys via Redis pipeline, computes weighted composite score (0-100) with labels Low/Moderate/Elevated/Severe/Critical
- Create server handler get-economic-stress.ts reading from economic:stress-index:v1
- Register economicStress in BOOTSTRAP_CACHE_KEYS (both cache-keys.ts and api/bootstrap.js) as slow tier
- Add gateway.ts cache tier entry (slow) for new RPC route
- Create EconomicStressPanel.ts: composite score header, gradient needle bar, 2x3 component grid with score bars, desktop notification on threshold cross (>=70, >=85)
- Wire economic-stress panel in panels.ts (all 4 variants), panel-layout.ts, and data-loader.ts
- Regenerate OpenAPI docs and TypeScript client/server types

* fix(economic-stress): null for missing FRED data + tech variant panel

- Add 'economic-stress' panel to TECH_PANELS defaults (was missing, only appeared in full/finance/commodity variants)
- Seed: write rawValue: null + missing: true when no valid FRED observation found, preventing zero-valued yield curve/bank spread readings from being conflated with missing data
- Proto: add missing bool field to EconomicStressComponent message; regenerate client/server types + OpenAPI docs
- Server handler: propagate missing flag from Redis; pass rawValue: 0 on wire when missing to satisfy proto double type
- Panel: guard on c.missing (not rawValue === 0) to show grey N/A card with no score bar for unavailable components

* fix(economic-stress): add purple Critical zone to gradient bar

Update gradient stops to match the 5 equal tier boundaries (0-20-40-60-80-100),
adding the #8e44ad purple stop at 80% so scores 80-100 render as Critical purple
instead of plain red.
2026-03-29 11:19:35 +04:00
Elie Habib
d01469ba9c feat(aviation): add SearchGoogleFlights and SearchGoogleDates RPCs (#2446)
* feat(aviation): add SearchGoogleFlights and SearchGoogleDates RPCs

Port Google Flights internal API from fli Python library as two new
aviation service RPCs routed through the Railway relay.

- Proto: SearchGoogleFlights and SearchGoogleDates messages and RPCs
- Relay: handleGoogleFlightsSearch and handleGoogleFlightsDates handlers
  with JSONP parsing, 61-day chunking for date ranges, cabin/stops/sort mappers
- Server handlers: forward params to relay google-flights endpoints
- gateway.ts: no-store for flights, medium cache for dates

* feat(mcp): expose search_flights and search_flight_prices_by_date tools

* test(mcp): update tool count to 24 after adding search_flights and search_flight_prices_by_date

* fix(aviation): address PR review issues in Google Flights RPCs

P1: airline filtering — use gfParseAirlines() in relay (handles comma-joined
  string from codegen) and parseStringArray() in server handlers

P1: partial chunk failure now sets degraded: true instead of silently
  returning incomplete data as success; relay includes partial: true flag

P2: round-trip date search validates trip_duration > 0 before proceeding;
  returns 400 when is_round_trip=true and duration is absent/zero

P2: relay mappers accept user-friendly aliases ('0'/'1' for max_stops,
  'price'/'departure' for sort_by) alongside symbolic enum values;
  MCP tool docs updated to match

* fix(aviation): use cachedFetchJson in searchGoogleDates for stampede protection

Medium cache tier (10 min) requires Redis-level coalescing to prevent
concurrent requests from all hitting the relay before cache warms.
Cache key includes all request params (sorted airlines for stable keys).

* fix(aviation): always use getAll() for airlines in relay; add multi-airline tests

The OR short-circuit (get() || getAll()) meant get() returned the first
airline value (truthy), so getAll() never ran and only one airline was
forwarded to Google. Fix: unconditionally use getAll().

Tests cover: multi-airline repeated params, single airline, empty array,
comma-joined string from codegen, partial degraded flag propagation.
2026-03-28 23:07:18 +04:00
Elie Habib
e43292b057 feat(market-implications): add transmission chain to implication cards (#2439)
Models the causal path from geopolitical event to price impact as a
2-4 node chain on each card. Each node has a label, impact_type, and
logic sentence. Nodes are tap/click-expandable inline.

- Proto: add TransmissionNode message + field 10 on MarketImplicationCard
- Seeder: prompt addition + VALID_IMPACT_TYPES validation (>=2 node gate)
- Handler: map transmission_chain snake_case -> transmissionChain camelCase
- Client: add normalizeCard() to handle both bootstrap (snake_case) and
  API (camelCase) paths; applied at both data entry points
- Renderer: renderChain() with arrow-separated nodes; delegated click
  handler collapses same node, replaces on different node click
- Tests: 4 cases covering snake/camel conversion, absent field default,
  handler backward-compat
2026-03-28 21:17:41 +04:00
Elie Habib
564c252d48 feat(panels): move FrameworkSelector to AI Market Implications panel (#2394)
* feat(panels): move FrameworkSelector from CountryDeepDivePanel to MarketImplicationsPanel

CountryDeepDivePanel was the only panel where the framework selector
wasn't connected to the AI generation path (the country-intel.ts service
reads the framework independently via getActiveFrameworkForPanel).
MarketImplicationsPanel is a better home for it.

Changes:
- Remove FrameworkSelector + hasPremiumAccess from CountryDeepDivePanel
- Add FrameworkSelector (panelId: 'market-implications') to MarketImplicationsPanel
  with note "Applies to next AI regeneration"
- Add 'market-implications' to AnalysisPanelId union type
- fetchMarketImplications() now accepts optional framework param and passes
  it as ?framework= query string to the API
- data-loader subscribes to 'market-implications' framework changes and
  re-triggers loadMarketImplications(), passing the active framework

* fix(market-implications): cache per-framework to prevent N×1 API calls

Previously: any active framework bypassed the client-side TTL cache
entirely, so 100 users with the same framework = 100 separate API calls
per 10-minute window.

Fix:
- Client: replace single-entry cache with Map<frameworkId, {data, cachedAt}>
  so each framework variant is cached separately for 10 min
- API: pass ?frameworkId= (stable ID, not the full systemPromptAppend text)
- Server: reads intelligence:market-implications:v1:{frameworkId} when a
  known framework ID is present; falls back to the default key if the
  framework-specific key doesn't exist yet
- Proto: add framework_id field (sebuf.http.query) to ListMarketImplicationsRequest

With this, 100 users sharing the same framework hit the server once per
10 min per unique frameworkId — the server's framework-keyed Redis entry
is shared across all of them.

* chore(proto): regenerate OpenAPI spec after ListMarketImplicationsRequest frameworkId field
2026-03-28 02:08:33 +04:00
Elie Habib
9a277233a0 fix(disease-outbreaks): use TGH exact lat/lng for map pins, fix density and location display (#2393)
* chore: redeploy to pick up WORLDMONITOR_VALID_KEYS fix

* fix(disease-outbreaks): use TGH lat/lng for map pins, fix location, add cases + date to tooltip

Root cause: all map pins collapsed to country centroids (getCountryCentroid) even though TGH
provides exact lat/lng per alert. Deduplication further collapsed all same-disease+country
alerts into one pin. 90-day lookback of ~1,600 TGH records was being reduced to ~10 pins.

- proto: add lat, lng, cases fields to DiseaseOutbreakItem (field 10/11/12)
- seed: preserve _lat/_lng/_cases from TGH bundle per alert
- seed: trim place_name to first comma-segment to avoid "Riga, Riga, Latvia" display
- seed: TGH items skip keyword filter (already disease-curated) and deduplication
- seed: raise cap to 150 TGH + 50 WHO/CDC/ONT (from flat 50 total)
- DeckGLMap: use item.lat/lng when non-zero, fall back to country centroid
- tooltip: add date and case count (when available) below source name
- sourceVersion bumped to v6

* chore: regenerate HealthService OpenAPI docs — add lat, lng, cases fields

* fix(mcp): address Greptile P2s — coordinate falsy coercion, dedup sort order, dead tooltip branch

- Use Number.isFinite() + null-coalescing (??) instead of || 0 for lat/lng; prevents
  treating equatorial coord 0 as "missing" (even though TGH already filters !rec.lat)
- DeckGLMap: use Number.isFinite + !== 0 guard instead of falsy && check for lat/lng
- Sort otherOutbreaks by publishedAt desc BEFORE deduplication so first-seen = most recent
- Remove redundant re-sort of dedupedOthers (already sorted above)
- Simplify metaHtml: always show date (publishedAt is always set); remove dead else-branch
2026-03-28 01:54:44 +04:00
Elie Habib
1f56afeb82 feat(panels): disease outbreaks panel/layer, social velocity panel, shipping stress tab (#2383)
* feat(panels): disease outbreaks panel/layer, social velocity panel, shipping stress tab

- DiseaseOutbreaksPanel: feed-style panel with alert/warning/watch filter pills, source links, relative timestamps (WHO/ProMED/HealthMap)
- SocialVelocityPanel: ranked Reddit trending posts by velocity score with subreddit badge, vote/comment counts, velocity bar
- SupplyChainPanel: Stress tab with composite stress gauge and carrier table with sparklines (GetShippingStressResponse)
- diseaseOutbreaks map layer: ScatterplotLayer via country centroids, color/radius by alert level, tooltip
- MapContainer.setDiseaseOutbreaks(): cached setter with DeckGLMap delegation
- data-loader: loadDiseaseOutbreaks/loadSocialVelocity/loadSupplyChain with stress wired into tasks
- MapLayers.diseaseOutbreaks added to types, layer registry (globe icon), full variant order, all default objects

🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.com/claude-code) + Compound Engineering v2.49.0

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(supply-chain): add upstreamUnavailable to ShippingStressResponse, restore test-compatible banner guard

* fix(panels): filter pills use alertLevel equality, sanitizeUrl on hrefs, globe TODO, E2E layer enabled

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-03-27 23:52:59 +04:00
Elie Habib
110ab402c4 feat(intelligence): analytical framework selector for AI panels (#2380)
* feat(frameworks): add settings section and import modal

- Add Analysis Frameworks group to preferences-content.ts between Intelligence and Media sections
- Per-panel active framework display (read-only, 4 panels)
- Skill library list with built-in badge, Rename and Delete actions for imported frameworks
- Import modal with two tabs: From agentskills.io (fetch + preview) and Paste JSON
- All error cases handled inline: network, domain validation, missing instructions, invalid JSON, duplicate name, instructions too long, rate limit
- Add api/skills/fetch-agentskills.ts edge function (proxy to agentskills.io)
- Add analysis-framework-store.ts (loadFrameworkLibrary, saveImportedFramework, deleteImportedFramework, renameImportedFramework, getActiveFrameworkForPanel)
- Add fw-* CSS classes to main.css matching dark panel aesthetic

* feat(panels): wire analytical framework store into InsightsPanel, CountryDeepDive, DailyMarketBrief, DeductionPanel

- InsightsPanel: append active framework to geoContext in updateFromClient(); subscribe in constructor, unsubscribe in destroy()
- CountryIntelManager: pass framework as query param to fetchCountryIntelBrief(); subscribe to re-open brief on framework change; unsubscribe in destroy()
- DataLoaderManager: add dailyBriefGeneration counter for stale-result guard; pass frameworkAppend to buildDailyMarketBrief(); subscribe to framework changes to force refresh; unsubscribe in destroy()
- daily-market-brief service: add frameworkAppend? field to BuildDailyMarketBriefOptions; append to extendedContext before summarize call
- DeductionPanel: append active framework to geoContext in handleSubmit() before RPC call

* feat(frameworks): add FrameworkSelector UI component

- Create FrameworkSelector component with premium/locked states
- Premium: select dropdown with all framework options, change triggers setActiveFrameworkForPanel
- Locked: disabled select + PRO badge, click calls showGatedCta(FREE_TIER)
- InsightsPanel: adds asterisk note (client-generated analysis hint)
- Wire into InsightsPanel, DailyMarketBriefPanel, DeductionPanel (via this.header)
- Wire into CountryDeepDivePanel header right-side (no Panel base, panel=null)
- Add framework-selector CSS to main.css

* fix(frameworks): make new proto fields optional in generated types

* fix(frameworks): extract firstMsg to satisfy strict null checks in tsconfig.api.json

* fix(docs): add blank lines around lists/headings to pass markdownlint

* fix(frameworks): add required proto string fields to call sites after make generate

* chore(review): add code review todos 041-057 for PR #2380

7 review agents (TypeScript, Security, Architecture, Performance,
Simplicity, Agent-Native, Learnings) identified 17 findings across
5 P1, 8 P2, and 4 P3 categories.
2026-03-27 23:36:44 +04:00
Elie Habib
e3bc79a19c fix(panels): yield curve title, macro EU tab polish, trade anomaly badge (#2381)
* fix(panels): yield curve title, macro EU tab polish, trade anomaly badge

YieldCurvePanel: rename title to "Yield Curve & Rates", rename tab to "US Curve" to disambiguate from ECB Rates tab

MacroTilesPanel EU tab: shorten verbose labels (HICP (YoY), Unemployment, GDP Growth (QoQ)), add fmtEuDate to show "Jan 2026" instead of raw YYYY-MM, wire priorValue from euAvg so "vs prior" delta renders on all 4 EU tiles

TradePolicyPanel: fix anomaly badge rendering as "RussiaAnomaly" — trade-anomaly-badge CSS class had no styles so text rendered inline; replaced with inline-styled red pill badge with proper spacing

seed-eurostat-country-data: bump lastTimePeriod 1→2 for CPI/GDP; update parseEurostatResponse to capture second observation as priorValue; pass priorValue through return value

* fix(panels): add prior_value to EurostatMetric proto, remove unsafe type casts, use CSS var for anomaly badge

- Add prior_value + has_prior fields to EurostatMetric proto (was missing, causing unsafe type cast)
- Run make generate to regenerate stubs (priorValue/hasPrior now properly typed)
- MacroTilesPanel: replace (m as {priorValue?:number}) cast with m?.hasPrior guard
- seed-eurostat-country-data: emit hasPrior: true when priorValue is available
- TradePolicyPanel: replace hardcoded #e74c3c with var(--red) CSS variable

Addresses Greptile review comments on PR #2381.
2026-03-27 23:29:55 +04:00
Elie Habib
1e1f377078 feat(panels): Disease Outbreaks, Shipping Stress, Social Velocity, nuclear test site enrichment (#2375)
* feat(panels): Disease Outbreaks, Shipping Stress, Social Velocity, nuclear test site monitoring

- Add HealthService proto with ListDiseaseOutbreaks RPC (WHO + ProMED RSS)
- Add GetShippingStress RPC to SupplyChainService (Yahoo Finance carrier ETFs)
- Add GetSocialVelocity RPC to IntelligenceService (Reddit r/worldnews + r/geopolitics)
- Enrich earthquake seed with Haversine nuclear test-site proximity scoring
- Add 5 nuclear test sites to NUCLEAR_FACILITIES (Punggye-ri, Lop Nur, Novaya Zemlya, Nevada NTS, Semipalatinsk)
- Add shipping stress + social velocity seed loops to ais-relay.cjs
- Add seed-disease-outbreaks.mjs Railway cron script
- Wire all new RPCs: edge functions, handlers, gateway cache tiers, health.js STANDALONE_KEYS/SEED_META

* fix(relay): apply gold standard retry/TTL-extend pattern to shipping-stress and social-velocity seeders

* fix(review): address all PR #2375 review findings

- health.js: shippingStress maxStaleMin 30→45 (3x interval), socialVelocity 20→30 (3x interval)
- health.js: remove shippingStress/diseaseOutbreaks/socialVelocity from ON_DEMAND_KEYS (relay/cron seeds, not on-demand)
- cache-keys.ts: add shippingStress, diseaseOutbreaks, socialVelocity to BOOTSTRAP_CACHE_KEYS
- ais-relay.cjs: stressScore formula 50→40 (neutral market = moderate, not elevated)
- ais-relay.cjs: fetchedAt Date.now() (consistent with other seeders)
- ais-relay.cjs: deduplicate cross-subreddit article URLs in social velocity loop
- seed-disease-outbreaks.mjs: WHO URL → specific DON RSS endpoint (not dead general news feed)
- seed-disease-outbreaks.mjs: validate() requires outbreaks.length >= 1 (reject empty array)
- seed-disease-outbreaks.mjs: stable id using hash(link) not array index
- seed-disease-outbreaks.mjs: RSS regexes use [\s\S]*? for CDATA multiline content
- seed-earthquakes.mjs: Lop Nur coordinates corrected (41.39,89.03 not 41.75,88.35)
- seed-earthquakes.mjs: sourceVersion bumped to usgs-4.5-day-nuclear-v1
- earthquake.proto: fields 8-11 marked optional (distinguish not-enriched from enriched=false/0)
- buf generate: regenerate seismology service stubs

* revert(cache-keys): don't add new keys to bootstrap without frontend consumers

* fix(panels): address all P1/P2/P3 review findings for PR #2375

- proto: add INT64_ENCODING_NUMBER annotation + sebuf import to get_shipping_stress.proto (run make generate)
- bootstrap: register shippingStress (fast), socialVelocity (fast), diseaseOutbreaks (slow) in api/bootstrap.js + cache-keys.ts
- relay: update WIDGET_SYSTEM_PROMPT with new bootstrap keys and live RPCs for health/supply-chain/intelligence
- seeder: remove broken ProMED feed URL (promedmail.org/feed/ returns HTML 404); add 500K size guard to fetchRssItems; replace private COUNTRY_CODE_MAP with shared geo-extract.mjs; remove permanently-empty location field; bump sourceVersion to who-don-rss-v2
- handlers: remove dead .catch from all 3 new RPC handlers; fix stressLevel fallback to low; fix fetchedAt fallback to 0
- services: add fetchShippingStress, disease-outbreaks.ts, social-velocity.ts with getHydratedData consumers
2026-03-27 22:33:45 +04:00
Elie Habib
9480b547d5 feat(feeds): US Natural Gas Storage weekly seeder (EIA NW2_EPG0_SWO_R48_BCF) (#2353)
* feat(feeds): US Natural Gas Storage seeder via EIA (NW2_EPG0_SWO_R48_BCF)

Adds weekly EIA natural gas working gas storage for the Lower-48 states
(series NW2_EPG0_SWO_R48_BCF, in Bcf), mirroring the crude inventories
pattern exactly. Companion dataset to EU gas storage (GIE AGSI+).

- proto: GetNatGasStorage RPC + NatGasStorageWeek message
- seed-economy.mjs: fetchNatGasStorage() in Promise.allSettled, writes
  economic:nat-gas-storage:v1 with 21-day TTL (3x weekly cadence)
- server handler: getNatGasStorage reads seeded key from Redis
- gateway: /api/economic/v1/get-nat-gas-storage → static tier
- health.js: BOOTSTRAP_KEYS + SEED_META (14-day maxStaleMin)
- bootstrap.js: KEYS + SLOW_KEYS
- cache-keys.ts: BOOTSTRAP_CACHE_KEYS + BOOTSTRAP_TIERS (slow)

* feat(feeds): add fetchNatGasStorageRpc consumer in economic service

Adds getHydratedData('natGasStorage') consumer required by bootstrap
key registry test, plus circuit breaker and RPC wrapper mirroring
fetchCrudeInventoriesRpc pattern.
2026-03-27 11:48:44 +04:00
Elie Habib
4438ef587f feat(feeds): ECB CISS European financial stress index (#2278) (#2334)
* feat(feeds): ECB CISS European financial stress seeder + GetEuFsi RPC (#2278)

- New seed-fsi-eu.mjs fetches ECB CISS (0-1 systemic stress index for Euro area)
  via SDMX-JSON REST API (free, no auth); TTL=604800s (7d, weekly data cadence)
- New GetEuFsi RPC in EconomicService with proto + handler; cache tier: slow
- FSIPanel now shows EU CISS gauge below US FSI with label thresholds:
  Low<0.2, Moderate<0.4, Elevated<0.6, High>=0.6
- Registered economic:fsi-eu:v1 in health.js BOOTSTRAP_KEYS + SEED_META,
  bootstrap.js, cache-keys.ts BOOTSTRAP_TIERS; hydrated via getHydratedData('euFsi')
- All 2348 test:data tests pass; typecheck + typecheck:api clean

* fix(ecb-ciss): address code review findings on PR #2334

- Raise FSI_EU_TTL from 604800s (7d) to 864000s (10d) to match other
  weekly seeds (bigmac, groceryBasket, fuelPrices) and provide a 3-day
  buffer against cron-drift or missed Saturday runs
- Format latestDate via toLocaleDateString() in FSIPanel CISS section
  instead of displaying the raw ISO string (e.g. "2025-04-04")

* fix(ecb-ciss): address Greptile review comments on PR #2334

- Fix misleading "Daily frequency" comment in seed-fsi-eu.mjs (SDMX
  uses 'D' series key but only Friday/weekly observations are present)
- Replace latestValue > 0 guards with Number.isFinite() in FSIPanel.ts
  so a valid CISS reading of exactly 0 is not incorrectly excluded

* chore: regenerate proto outputs after rebase
2026-03-27 11:07:17 +04:00
Elie Habib
1d0846fa98 feat(feeds): ECB Euro Area yield curve seeder (#2276) (#2330)
* feat(feeds): ECB Euro Area yield curve seeder — EU complement to US Treasury curve (#2276)

- Add scripts/seed-yield-curve-eu.mjs: fetches ECB AAA sovereign spot rates (1Y-30Y) via ECB Data Portal SDMX-JSON, writes to economic:yield-curve-eu:v1, TTL=259200s (3x daily interval)
- Add proto/worldmonitor/economic/v1/get_eu_yield_curve.proto + GetEuYieldCurve RPC
- Add server/worldmonitor/economic/v1/get-eu-yield-curve.ts handler (reads from Redis cache)
- Wire GetEuYieldCurve into handler.ts and gateway.ts (daily cache tier)
- Update YieldCurvePanel.ts: fetches EU curve in parallel with US FRED data, overlays green dashed line on chart with ECB AAA legend entry
- Update api/health.js: add euYieldCurve to BOOTSTRAP_KEYS + SEED_META (maxStaleMin=2880, daily seed)
- Re-run buf generate to update generated client/server TypeScript types

* fix(generated): restore @ts-nocheck in economic generated files after buf regeneration

* fix(ecb-yield-curve): address Greptile review comments on PR #2330

Include priorValues in allValues for yMin/yMax scale computation so
prior US curve polyline is never clipped outside the chart area.

* fix(generated): regenerate OpenAPI docs after rebase conflict resolution

Re-run make generate to fix schema ordering in EconomicService.openapi.yaml
and .json after manual conflict resolution introduced ordering inconsistencies.

* chore: regenerate proto outputs after rebase
2026-03-27 10:58:29 +04:00
Elie Habib
b6847e5214 feat(feeds): GIE AGSI+ EU gas storage seeder (#2281) (#2339)
* feat(feeds): GIE AGSI+ EU gas storage seeder — European energy security indicator (#2281)

- New scripts/seed-gie-gas-storage.mjs: fetches EU aggregate gas storage fill % from GIE AGSI+ API, computes 1-day change, trend (injecting/withdrawing/stable), and days-of-consumption estimate; TTL=259200s (3x daily); isMain guard; CHROME_UA; validates fillPct in (0,100]; graceful degradation when GIE_API_KEY absent
- New proto/worldmonitor/economic/v1/get_eu_gas_storage.proto + GetEuGasStorage RPC wired into EconomicService
- New server/worldmonitor/economic/v1/get-eu-gas-storage.ts handler (reads seeded Redis key)
- api/health.js: BOOTSTRAP_KEYS + SEED_META (maxStaleMin=2880, 2x daily cadence)
- api/bootstrap.js: euGasStorage key in SLOW_KEYS bucket
- Regenerated src/generated/ + docs/api/ via make generate

* fix(feeds): wire euGasStorage into cache-keys, gateway tier, and test PENDING_CONSUMERS

- server/_shared/cache-keys.ts: add euGasStorage → economic:eu-gas-storage:v1 (slow tier)
- server/gateway.ts: add /api/economic/v1/get-eu-gas-storage → slow RPC_CACHE_TIER
- tests/bootstrap.test.mjs: add euGasStorage to PENDING_CONSUMERS (no frontend panel yet)

* fix(gie-gas-storage): normalize seededAt to string to match proto int64 contract

Proto int64 seeded_at maps to string in JS; seed was writing Date.now() (number).
Fix seed to write String(Date.now()) and add handler-side normalization for any
stale Redis entries that may have the old numeric format.

* fix(feeds): coerce nullable fillPctChange1d/gasDaysConsumption to 0 (#2281)

Greptile P1: both fields could be null (single data-point run or missing
volume) but the proto interface declares them as non-optional numbers.
Seed script now returns 0 instead of null; handler defensively coerces
nulls from older cached blobs via nullish coalescing. Dead null-guard on
trend derivation also removed.
2026-03-27 10:50:20 +04:00
Elie Habib
c69a13a1a0 feat(feeds): Eurostat per-country economic data seeder (#2282) (#2340)
* feat(feeds): Eurostat per-country CPI/unemployment/GDP seeder for 10 EU member states (#2282)

* fix(feeds): address Greptile P1/P2 issues in eurostat seed parser

- P1: fix sparse-index iteration bug in parseEurostatResponse; loop now
  iterates over Object.keys(values) directly instead of using key count
  as sequential bound, correctly handling non-zero-based sparse indexes
- P2: remove unused dimIdx, geoStride, timeStride, totalSize variables
- P2: batch country fetches (3 at a time) to reduce peak Eurostat
  concurrency from 30 to 9 simultaneous requests

* chore: regenerate EconomicService openapi JSON after rebase
2026-03-27 10:26:11 +04:00
Elie Habib
b5faffb341 feat(feeds): ECB reference FX rates seeder (#2280) (#2337)
* feat(feeds): ECB daily reference FX rates seeder -- EUR/USD/GBP/JPY/CHF (#2280)

- New scripts/seed-ecb-fx-rates.mjs: fetches EUR/USD/GBP/JPY/CHF/CAD/AUD/CNY daily from ECB Data Portal (no API key required)
- New getEcbFxRates RPC in economic service (proto + handler + gateway slow tier)
- api/health.js: ecbFxRates in BOOTSTRAP_KEYS + SEED_META (2880min maxStale)
- api/bootstrap.js: ecbFxRates registered as SLOW_KEYS bootstrap entry
- TTL: 259200s (3x daily interval), isMain guard, CHROME_UA, validate>=3 pairs

* fix(feeds): register ecbFxRates in cache-keys.ts + add frontend hydration consumer

* fix(ecb-fx-rates): use dynamic CURRENCY dim position in series key parsing

CURRENCY is at index 1 in EXR series keys (FREQ:CURRENCY:CURRENCY_DENOM:EXR_TYPE:EXR_SUFFIX).
Using hardcoded keyParts[0] always read the FREQ field (always "0"), so all
series resolved to currencyCodes[0] (AUD) and only 1 pair was written to Redis
instead of all 7. Fix: find CURRENCY position via findIndex() and use
keyParts[currencyDimPos] to extract the correct per-series currency index.

* fix(feeds): hoist obs-dimension lookup + fix ECB UTC publication time

- Hoist obsPeriods/timeDim/timeValues above the series loop (loop-invariant, P2)
- Fix updatedAt timestamp: ECB publishes at 16:00 CET = 14:00 UTC (not 16:00 UTC), use T14:00:00Z (P2)

* chore(generated): fix EcbFxRate schema ordering in EconomicService openapi.json

buf generate places EcbFxRate alphabetically before EconomicEvent; align
committed file with code-generator output so proto freshness hook passes.
2026-03-27 10:11:47 +04:00
Elie Habib
f3b0280227 feat(economic): EIA weekly crude oil inventory seeder (#2142) (#2168)
* feat(economic): EIA weekly crude oil inventory seeder (#2142)

- scripts/seed-economy.mjs: add fetchCrudeInventories() fetching WCRSTUS1, compute weeklyChangeMb, write economic:crude-inventories:v1 (10-day TTL)
- proto/worldmonitor/economic/v1/get_crude_inventories.proto: new proto with CrudeInventoryWeek and GetCrudeInventories RPC
- server/worldmonitor/economic/v1/get-crude-inventories.ts: RPC handler reading seeded key with getCachedJson(..., true)
- server/worldmonitor/economic/v1/handler.ts: wire in getCrudeInventories
- server/gateway.ts: add static cache tier for /api/economic/v1/get-crude-inventories
- api/health.js: crudeInventories in BOOTSTRAP_KEYS + SEED_META (maxStaleMin: 20160, 2x weekly cadence)
- src/services/economic/index.ts: add fetchCrudeInventoriesRpc() with circuit breaker
- src/components/EnergyComplexPanel.ts: surface 8-week sparkline and WoW change in energy panel
- src/app/data-loader.ts: call fetchCrudeInventoriesRpc() in loadOilAnalytics()

* fix: remove stray market-implications gateway entry from crude-inventories branch

* fix(crude-inventories): address ce-review P1/P2 findings before merge

- api/bootstrap.js: register crudeInventories in BOOTSTRAP_CACHE_KEYS + SLOW_KEYS (P1-001)
- server/_shared/cache-keys.ts: add crudeInventories key + tier to match bootstrap.js
- api/health.js: remove bundled marketImplications (belongs in separate PR) (P1-002)
- src/services/economic/index.ts: add isFeatureAvailable('energyEia') gate (P2-003)
- src/services/economic/index.ts: use getHydratedData('crudeInventories') on first load
- proto/get_crude_inventories.proto: weekly_change_mb → optional double (P2-004)
- scripts/seed-economy.mjs: CRUDE_INVENTORIES_TTL 10d → 21d (3× cadence) (P2-005)
- scripts/seed-economy.mjs: period format validation with YYYY-MM-DD regex (P3-007)
- src/app/data-loader.ts: warn on crude fetch rejection (P2-006)

* fix(crude-inventories): schema validation, MIN_ITEMS gate, handler logging, raw=true docs

- Handler: document raw=true param, log errors instead of silent catch
- Seeder: CRUDE_MIN_WEEKS=4 guard prevents quota-hit empty writes
- Seeder: isValidWeek() schema validation before Redis write

* chore: regenerate openapi docs after rebase (adds getCrudeInventories + getEconomicCalendar)

* fix(gateway): add list-market-implications to RPC_CACHE_TIER

* chore: exclude todos/ from markdownlint
2026-03-27 09:42:26 +04:00
Elie Habib
d80736d1ba feat(heatmap): add sorted bar chart view to sector HeatmapPanel (#2326)
* feat(heatmap): add sorted bar chart view to HeatmapPanel (#2246)

- Add FearGreedSectorPerformance message to proto + regenerate
- Expose sectorPerformance array in GetFearGreedIndexResponse RPC handler
- HeatmapPanel.renderHeatmap accepts optional sectorBars; renders
  sorted horizontal bar chart below existing tile grid
- Sectors ranked by day change (gainers first, losers last)
- Green/red bars proportional to |change| (max ~3% = 100% width)
- Reuses fearGreedIndex bootstrap data — zero extra network calls
- Add .heatmap-bar-chart CSS: 18px rows, 10px font, compact layout

* fix(heatmap): address code review P1 findings on PR #2326

- Server: guard change1d with Number.isFinite fallback so non-numeric
  Redis values (NaN) coerce to 0 instead of propagating through
- Client: filter out any NaN change1d entries before computing maxAbs;
  Math.max propagates NaN so a single bad entry makes all bars invisible
  (width:NaN%); also bail early if filtered list is empty

* fix(heatmap): address Greptile review comments on PR #2326

Use var(--green)/var(--red) for bar fill colour to match text label
CSS variables and avoid visual mismatch across themes.
2026-03-27 09:13:51 +04:00
Elie Habib
d4917cf00b feat(military): USPTO PatentsView defense/dual-use patent filing seeder (#2047) (#2091)
* feat(military): USPTO PatentsView defense/dual-use patent seeder (#2047)

* fix(military): correct PatentsView API field names, sort param, and total semantics

- Use nested dot-notation field names required by PatentsView v1 API:
  assignees.assignee_organization (was flat assignee_organization)
  cpc_at_issue.cpc_subclass_id (was non-existent cpc_subgroup_id)
- Split sort into separate s= param; fix per_page -> size in o= param
  (sort inside o= and per_page are both rejected by PatentsView v1)
- Fix listDefensePatents total to reflect full seeded dataset size,
  not the post-filter count (proto doc: "Total number of filings in
  the seeded dataset")

* feat(defense-patents): add DefensePatentsPanel — R&D Signal frontend

Panel displays weekly USPTO defense/dual-use patent filings from seed.
Five tabs: All | Comms (H04B) | Semiconductors (H01L) | Ammunition (F42B)
| AI (G06N) | Biotech (C12N). Shows assignee, title, CPC tag, date, USPTO link.

Wired into panel-layout.ts and registered as 'defense-patents' in panels.ts.

* fix(defense-patents): TTL 3x interval, recordCount, maxStaleMin 2x, wire into App.ts

- CACHE_TTL: 7d → 21d (3× weekly interval) so key survives missed runs
- recordCount: add (d) => d?.patents?.length ?? 0 so seed logs correct count
- health.js maxStaleMin: 10080 → 20160 (2× 7-day interval per gold standard)
- App.ts: import DefensePatentsPanel, add primeTask (initial load) and
  scheduleRefresh at 24h interval (daily poll for weekly data)
2026-03-26 17:50:37 +04:00
Elie Habib
2939b1f4a1 feat(finance-panels): add 7 macro/market panels + Daily Brief context (issues #2245-#2253) (#2258)
* feat(fear-greed): add regime state label, action stance badge, divergence warnings

Closes #2245

* feat(finance-panels): add 7 new finance panels + Daily Brief macro context

Implements issues #2245 (F&G Regime), #2246 (Sector Heatmap bars),
#2247 (MacroTiles), #2248 (FSI), #2249 (Yield Curve), #2250 (Earnings
Calendar), #2251 (Economic Calendar), #2252 (COT Positioning),
#2253 (Daily Brief prompt extension).

New panels:
- MacroTilesPanel: CPI YoY, Unemployment, GDP, Fed Rate tiles via FRED
- FSIPanel: Financial Stress Indicator gauge (HYG/TLT/VIX/HY-spread)
- YieldCurvePanel: SVG yield curve chart with inverted/normal badge
- EarningsCalendarPanel: Finnhub earnings calendar with BMO/AMC/BEAT/MISS
- EconomicCalendarPanel: FOMC/CPI/NFP events with impact badges
- CotPositioningPanel: CFTC disaggregated COT positioning bars
- MarketPanel: adds sorted bar chart view above sector heatmap grid

New RPCs:
- ListEarningsCalendar (market/v1)
- GetCotPositioning (market/v1)
- GetEconomicCalendar (economic/v1)

Seed scripts:
- seed-earnings-calendar.mjs (Finnhub, 14-day window, TTL 12h)
- seed-economic-calendar.mjs (Finnhub, 30-day window, TTL 12h)
- seed-cot.mjs (CFTC disaggregated text file, TTL 7d)
- seed-economy.mjs: adds yield curve tenors DGS1MO/3MO/6MO/1/2/5/30
- seed-fear-greed.mjs: adds FSI computation + sector performance

Daily Brief: extends buildDailyMarketBrief with optional regime,
yield curve, and sector context fed to the LLM summarization prompt.

All panels default enabled in FINANCE_PANELS, disabled in FULL_PANELS.

🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.ai/claude-code) + Compound Engineering v2.40.0

Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>

* fix(finance-panels): address code review P1/P2 findings

P1 - Security/Correctness:
- EconomicCalendarPanel: add escapeHtml on all 7 Finnhub-sourced fields
- EconomicCalendarPanel: fix panel contract (public fetchData():boolean,
  remove constructor self-init, add retry callbacks to all showError calls)
- YieldCurvePanel: fix NaN in xPos() when count <= 1 (divide-by-zero)
- seed-earnings-calendar: move Finnhub API key from URL to X-Finnhub-Token header
- seed-economic-calendar: move Finnhub API key from URL to X-Finnhub-Token header
- seed-earnings-calendar: add isMain guard around runSeed() call
- health.js + bootstrap.js: register earningsCalendar, econCalendar, cotPositioning keys
- health.js dataSize(): add earnings + instruments to property name list

P2 - Quality:
- FSIPanel: change !resp.fsiValue → resp.fsiValue <= 0 (rejects valid zero)
- data-loader: fix Promise.allSettled type inference via indexed destructure
- seed-fear-greed: allowlist cnnLabel against known values before writing to Redis
- seed-economic-calendar: remove unused sleep import
- seed-earnings-calendar + econ-calendar: increase TTL 43200 → 129600 (36h = 3x interval)
- YieldCurvePanel: use SERIES_IDS const in RPC call (single source of truth)

* fix(bootstrap): remove on-demand panel keys from bootstrap.js

earningsCalendar, econCalendar, cotPositioning panels fetch via RPC
on demand — they have no getHydratedData consumer in src/ and must
not be in api/bootstrap.js. They remain in api/health.js BOOTSTRAP_KEYS
for staleness monitoring.

* fix(compound-engineering): fix markdown lint error in local settings

* fix(finance-panels): resolve all P3 code-review findings

- 030: MacroTilesPanel: add `deltaFormat?` field to MacroTile interface,
  define per-tile delta formatters (CPI pp, GDP localeString+B),
  replace fragile tile.id switch in tileHtml with fmt = deltaFormat ?? format
- 031: FSIPanel: check getHydratedData('fearGreedIndex') at top of
  fetchData(); extract fsi/vix/hySpread from headerMetrics and render
  synchronously; fall back to live RPC only when bootstrap absent
- 032: All 6 finance panels: extract lazy module-level client singletons
  (EconomicServiceClient or MarketServiceClient) so the client is
  constructed at most once per panel module lifetime, not on every fetchData
- 033: get-fred-series-batch: add BAMLC0A0CM and SOFR to ALLOWED_SERIES
  (both seeded by seed-economy.mjs but previously unreachable via RPC)

* fix(finance-panels): health.js SEED_META, FSI calibration, seed-cot catch handler

- health.js: add SEED_META entries for earningsCalendar (1440min), econCalendar
  (1440min), cotPositioning (14400min) — without these, stopped seeds only alarm
  CRIT:EMPTY after TTL expiry instead of earlier WARN:STALE_SEED
- seed-cot.mjs: replace bare await with .catch() handler consistent with other seeds
- seed-fear-greed.mjs: recalibrate FSI thresholds to match formula output range
  (Low>=1.5, Moderate>=0.8, Elevated>=0.3; old values >=0.08/0.05/0.03 were
  calibrated for [0,0.15] but formula yields ~1-2 in normal conditions)
- FSIPanel.ts: fix gauge fillPct range to [0, 2.5] matching recalibrated thresholds
- todos: fix MD022/MD032 markdown lint errors in P3 review files

---------

Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
2026-03-26 08:03:09 +04:00
Elie Habib
45469fae3b feat(forecasts): NEXUS panel redesign + simulation theater data via theaterSummariesJson (#2244)
Redesigns ForecastPanel with a theater-first NEXUS layout that surfaces simulation
outcomes alongside forecast probabilities in a unified view. Adds the
theaterSummariesJson field to the GetSimulationOutcome proto so the server can return
pre-condensed UI data from a single Redis read without any additional R2 fetches.

- proto: add theater_summaries_json = 9 to GetSimulationOutcomeResponse
- seed-forecasts.mjs: embed condensed uiTheaters array in Redis pointer at write time
- get-simulation-outcome.ts: serialize pointer.uiTheaters → theaterSummariesJson in RPC response
- src/services/forecast.ts: add fetchSimulationOutcome() returning theaterSummariesJson string
- src/app/data-loader.ts: load simulation outcome alongside forecasts, call updateSimulation()
- ForecastPanel.ts: full NEXUS redesign with SVG circular gauges, expandable theater detail,
  compact prob table, CSS custom property for per-theater accent color, race condition guard
  (skip render when forecasts array still empty on simulation arrival)
2026-03-25 22:44:48 +04:00
Elie Habib
01f6057389 feat(simulation): MiroFish Phase 2 — theater-limited simulation runner (#2220)
* feat(simulation): MiroFish Phase 2 — theater-limited simulation runner

Adds the simulation execution layer that consumes simulation-package.json
and produces simulation-outcome.json for maritime chokepoint + energy/logistics
theaters, closing the WorldMonitor → MiroFish handoff loop.

Changes:
- scripts/seed-forecasts.mjs: 2-round LLM simulation runner (prompt builders,
  JSON extractor, runTheaterSimulation, writeSimulationOutcome, task queue
  with NX dedup lock, runSimulationWorker poll loop)
- scripts/process-simulation-tasks.mjs: standalone worker entry point
- proto: GetSimulationOutcome RPC + make generate
- server/worldmonitor/forecast/v1/get-simulation-outcome.ts: RPC handler
- server/gateway.ts: slow tier for get-simulation-outcome
- api/health.js: simulationOutcomeLatest in STANDALONE + ON_DEMAND keys
- tests: 14 new tests for simulation runner functions

* fix(simulation): address P1/P2 code review findings from PR #2220

Security (P1 #018):
- sanitizeForPrompt() applied to all entity/seed fields interpolated into
  Round 1 prompt (entityId, class, stance, seedId, type, timing)
- sanitizeForPrompt() applied to actorId and entityIds in Round 2 prompt
- sanitizeForPrompt() + length caps applied to all LLM array fields written
  to R2 (dominantReactions, stabilizers, invalidators, keyActors, timingMarkers)

Validation (P1 #019):
- Added validateRunId() regex guard
- Applied in enqueueSimulationTask() and processNextSimulationTask() loop

Type safety (P1 #020):
- Added isOutcomePointer() and isPackagePointer() type guards in TS handlers
- Replaced unsafe as-casts with runtime-validated guards in both handlers

Correctness (P2 #022):
- Log warning when pkgPointer.runId does not match task runId

Architecture (P2 #024):
- isMaritimeChokeEnergyCandidate() accepts both flat and nested topBucketId
- Call site simplified to pass theater directly

Performance (P2 #025):
- SIMULATION_ROUND1_MAX_TOKENS raised 1800 to 2200
- Added max 3 initialReactions instruction to Round 1 prompt

Maintainability (P2 #026):
- Simulation pointer keys exported from server/_shared/cache-keys.ts
- Both TS handlers import from shared location

Documentation (P2 #027):
- Strengthened runId no-op description in proto and OpenAPI spec

* fix(todos): add blank lines around lists in markdown todo files

* style(api): reformat openapi yaml to match linter output

* test(simulation): add flat-shape filter test + getSimulationOutcome handler coverage

Two tests identified as missing during PR #2220 review:

1. isMaritimeChokeEnergyCandidate flat-shape tests — covers the || candidate.topBucketId
   normalization added in the P1/P2 review pass. The existing tests only used the nested
   marketContext.topBucketId shape; this adds the flat root-field shape that arrives from
   the simulation-package.json JSON (selectedTheaters entries have topBucketId at root).

2. getSimulationOutcome handler structural tests — verifies the isOutcomePointer guard,
   found:false NOT_FOUND return, found:true success path, note population on runId mismatch,
   and redis_unavailable error string. Follows the readSrc static-analysis pattern used
   elsewhere in server-handlers.test.mjs (handler imports Redis so full integration test
   would require a test Redis instance).
2026-03-25 13:55:59 +04:00
Elie Habib
e548e6cca5 feat(intelligence): cross-source signal aggregator with composite escalation (#2143) (#2164)
* feat(intelligence): cross-source signal aggregator with composite escalation (#2143)

Adds a threshold-based signal aggregator seeder that reads 15+ already-seeded
Redis keys every 15 minutes, ranks cross-domain signals by severity, and detects
composite escalation when >=3 signal categories co-fire in the same theater.

* fix(cross-source-signals): wire panel data loading, inline styles, seeder cleanup

- New src/services/cross-source-signals.ts: fetch via IntelligenceServiceClient with circuit breaker
- data-loader.ts: add loadCrossSourceSignals() + startup batch entry (SITE_VARIANT !== 'happy' guard)
- App.ts: add primeVisiblePanelData entry + scheduleRefresh at 15min interval
- base.ts: add crossSourceSignals: 15 * 60 * 1000 to REFRESH_INTERVALS
- CrossSourceSignalsPanel.ts: replace all CSS class usage with inline styles (MarketImplicationsPanel pattern)
- seed-cross-source-signals.mjs: remove dead isMain var, fix afterPublish double-write, deterministic signal IDs, GDELT per-topic tone keys (military/nuclear/maritime) with 3-point declining trend + < -1.5 threshold per spec, bundled topics fallback

* fix(cross-source-signals): complete bootstrap wiring, seeder fixes, cmd-k entry

- cache-keys.ts: add crossSourceSignals to BOOTSTRAP_CACHE_KEYS + BOOTSTRAP_TIERS (slow)
- bootstrap.js: add crossSourceSignals key + SLOW_KEYS entry
- cross-source-signals.ts: add getHydratedData('crossSourceSignals') bootstrap hydration
- seed script: fix isDeclinig typo, maritime theater ternary (Global->Indo-Pacific), displacement year dynamic
- commands.ts: add panel:cross-source-signals to cmd-k

* feat(cross-source-signals): redesign panel — severity bars, filled badges, icons, theater pills

- 4px severity accent bar on all signal rows (scannable without reading badges)
- Filled severity badges: CRITICAL=solid red/white, HIGH=faint red bg, MED=faint yellow bg
- Type badge emoji prefix:  composite, 🔴 geo-physical, 📡 EW, ✈️ military, 📊 market, ⚠️ geopolitical
- Composite card: full glow (box-shadow) instead of 3px left border only
- Theater pill with inline age: "Middle East · 8m ago"
- Contributor pills: individual chips instead of dot-separated string
- Pulsing dot on composite escalation banner

* fix(cross-source-signals): code review fixes — module-level Sets, signal cap, keyframe scoping, OREF expansion

- Replace per-call Array literals in list-cross-source-signals.ts with module-level Set constants for O(1) lookups
- Add index-based fallback ID in normalizeSignal to avoid undefined ids
- Remove unused military:flights:stale:v1 from SOURCE_KEYS
- Add MAX_SIGNALS=30 cap before writing to Redis
- Expand extractOrefAlertCluster to any "do not travel" advisory (not just Israel)
- Add BASE_WEIGHT inline documentation explaining scoring scale
- Fix animation keyframe: move from setContent() <style> block to constructor (injected once), rename to cross-source-pulse-dot
- Fix GDELT extractor to read per-topic gdelt:intel:tone:{topic} keys with correct decline logic
- Fix isDeclinig typo, maritime dead ternary, and displacement year reference
2026-03-24 23:18:31 +04:00
Elie Habib
f87c8c71c4 feat(forecast): Phase 2 simulation package read path (#2219)
* feat(forecast): Phase 2 simulation package read path (getSimulationPackage RPC + Redis existence key)

- writeSimulationPackage now writes forecast:simulation-package:latest to Redis after
  successful R2 write, containing { runId, pkgKey, schemaVersion, theaterCount, generatedAt }
  with TTL matching TRACE_REDIS_TTL_SECONDS (60 days)
- New getSimulationPackage RPC handler reads Redis key, returns pointer metadata without
  requiring an R2 fetch (zero R2 cost for existence check)
- Wired into ForecastServiceHandler and server/gateway.ts cache tier (medium)
- Proto: GetSimulationPackage RPC + get_simulation_package.proto message definitions
- api/health.js: simulationPackageLatest added to STANDALONE_KEYS + ON_DEMAND_KEYS
- Tests: SIMULATION_PACKAGE_LATEST_KEY constant + writeSimulationPackage null-guard test

Closes todo #017 (Phase 2 prerequisites for MiroFish integration)

* chore(generated): regenerate proto types for GetSimulationPackage RPC

* fix(simulation-rpc): distinguish Redis failure from not-found; signal runId mismatch

- Add `error` field to GetSimulationPackageResponse: populated with
  "redis_unavailable" on Redis errors so callers can distinguish a
  healthy not-found (found=false, error="") from a Redis failure
  (found=false, error="redis_unavailable"). Adds console.warn on error.
- Add `note` field: populated when req.runId is supplied but does not
  match the latest package's runId, signalling that per-run filtering
  is not yet active (Phase 3).
- Add proto comment on run_id: "Currently ignored; reserved for Phase 3"
- Add milliseconds annotation to generated_at description.
- Simplify handler: extract NOT_FOUND constant, remove SimulationPackagePointer
  interface, remove || '' / || 0 guards on guaranteed-present fields.
- Regenerate all buf-generated files.

Fixes todos #018 (runId silently ignored) and #019 (error indistinguishable
from not-found). Also resolves todos #022 (simplifications) and #023
(OpenAPI required fields / generatedAt unit annotation).

* fix(simulation-rpc): change cache tier from medium to slow (aligns with deep-run update frequency)

* fix(simulation-rpc): fix key prefixing, make Redis errors reachable, no-cache not-found

Three P1 regressions caught in external review:

1. Key prefix bug: getCachedJson() applies preview:<sha>: prefix in non-production
   environments, but writeSimulationPackage writes the raw key via a direct Redis
   command. In preview/dev the RPC always returned found:false even when the package
   existed. Fix: new getRawJson() in redis.ts always uses the unprefixed key AND throws
   on failure instead of swallowing errors.

2. redis_unavailable unreachable: getCachedJson swallows fetch failures and missing-
   credentials by returning null, so the catch block for redis_unavailable was dead
   code. getRawJson() throws on HTTP errors and missing credentials, making the
   error: "redis_unavailable" contract actually reachable.

3. Negative-cache stampede: slow tier caches every 200 GET. A request before any deep
   run wrote a package returned { found:false } which the CDN cached for up to 1h,
   breaking post-run discovery. Fix: markNoCacheResponse() on both not-found and
   error paths so they are served fresh on every request.
2026-03-24 22:45:22 +04:00
Elie Habib
7013b2f9f1 feat(market): Fear & Greed Index 2.0 — 10-category composite sentiment panel (#2181)
* Add Fear & Greed Index 2.0 reverse engineering brief

Analyzes the 10-category weighted composite (Sentiment, Volatility,
Positioning, Trend, Breadth, Momentum, Liquidity, Credit, Macro,
Cross-Asset) with scoring formulas, data source audit, and
implementation plan for building it as a worldmonitor panel.

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* Add seed script implementation plan to F&G brief

Details exact endpoints, Yahoo symbols (17 calls), Redis key schema,
computed metrics, FRED series to add (BAMLC0A0CM, SOFR), CNN/AAII
sources, output JSON schema, and estimated runtime (~8s per seed run).

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* Update brief: all sources are free, zero paid APIs needed

- CBOE CDN CSVs for put/call ratios (totalpc.csv, equitypc.csv)
- CNN dataviz API for Fear & Greed (production.dataviz.cnn.io)
- Yahoo Finance for VIX9D/VIX3M/SKEW/RSP/NYA (standard symbols)
- FRED for IG spread (BAMLC0A0CM) and SOFR (add to existing array)
- AAII scrape for bull/bear survey (only medium-effort source)
- Breadth via RSP/SPY divergence + NYSE composite (no scraping)

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* Add verified Yahoo symbols for breadth + finalized source list

New discoveries:
- ^MMTH = % stocks above 200 DMA (direct Yahoo symbol!)
- C:ISSU = NYSE advance/decline data
- CNN endpoint accepts date param for historical data
- CBOE CSVs have data back to 2003
- 33 total calls per seed run, ~6s runtime

All 10 categories now have confirmed free sources.

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* Rewrite F&G brief as forward-looking design doc

Remove all reverse-engineering language, screenshot references, and
discovery notes. Clean structure: goal, scoring model, data sources,
formulas, seed script plan, implementation phases, MVP path.

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* docs: apply gold standard corrections to fear-greed-index-2.0 brief

* feat(market): add Fear & Greed Index 2.0 — 10-category composite sentiment panel

Composite 0-100 index from 10 weighted categories: sentiment (CNN F&G,
AAII, crypto F&G), volatility (VIX, term structure), positioning (P/C
ratio, SKEW), trend (SPX vs MAs), breadth (% >200d, RSP/SPY divergence),
momentum (sector RSI, ROC), liquidity (M2, Fed BS, SOFR), credit (HY/IG
spreads), macro (Fed rate, yield curve, unemployment), cross-asset
(gold/bonds/DXY vs equities).

Data layer:
- seed-fear-greed.mjs: 19 Yahoo symbols (150ms gaps), CBOE P/C CSVs,
  CNN F&G API, AAII scrape (degraded-safe), FRED Redis reads. TTL 64800s.
- seed-economy.mjs: add BAMLC0A0CM (IG spread) and SOFR to FRED_SERIES.
- Bootstrap 4-file checklist: cache-keys, bootstrap.js, health.js, handler.

Proto + RPC:
- get_fear_greed_index.proto with FearGreedCategory message.
- get-fear-greed-index.ts handler reads seeded Redis data.

Frontend:
- FearGreedPanel with gauge, 9-metric header grid, 10-category breakdown.
- Self-loading via bootstrap hydration + RPC fallback.
- Registered in panel-layout, App.ts (prime + refresh), panel config,
  Cmd-K commands, finance variant, i18n (en/ar/zh/es).

* fix(market): add RPC_CACHE_TIER entry for get-fear-greed-index

* fix(docs): escape bare angle bracket in fear-greed brief for MDX

* fix(docs): fix markdown lint errors in fear-greed brief (blank lines around headings/lists)

* fix(market): fix seed-fear-greed bugs from code review

- fredLatest/fredNMonthsAgo: guard parseFloat with Number.isFinite to
  handle FRED's "." missing-data sentinel (was returning NaN which
  propagated through scoring as a truthy non-null value)
- Remove 3 unused Yahoo symbols (^NYA, HYG, LQD) that were fetched
  but not referenced in any scoring category (saves ~450ms per run)
- fedRateStr: display effective rate directly instead of deriving
  target range via (fedRate - 0.25) which was incorrect

* fix(market): address P2/P3 review findings in Fear & Greed

- FearGreedPanel: add mapSeedPayload() to correctly map raw seed
  JSON to proto-shaped FearGreedData; bootstrap hydration was always
  falling through to RPC because seed shape (composite.score) differs
  from proto shape (compositeScore)
- FearGreedPanel: fix fmt() — remove === 0 guard and add explicit
  > 0 checks on VIX and P/C Ratio display to handle proto default
  zeros without masking genuine zero values (e.g. pctAbove200d)
- seed-fear-greed: remove broken history write — each run overwrote
  the key with a single-entry array (no read-then-append), making the
  90-day TTL meaningless; no consumer exists yet so defer to later
- seed-fear-greed: extract hySpreadVal const to avoid double fredLatest call
- seed-fear-greed: fix stale comment (19 symbols → 16 after prior cleanup)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-24 09:45:59 +04:00
Elie Habib
a1c3c1d684 feat(panels): AI Market Implications — LLM trade signals from live world state (#2146) (#2165)
* fix(intelligence): use camelCase field names for ListMarketImplicationsResponse

* fix(bootstrap): register marketImplications in cache-keys.ts and add hydration consumer

* chore: stage all market-implications feature files for proto freshness check

* feat(market-implications): add LLM routing env vars for market implications stage

* fix(market-implications): move types to services layer to fix boundary violation

* fix: add list-market-implications gateway tier entry

* fix(market-implications): add health.js entries + i18n tooltip key

- api/health.js: add marketImplications to BOOTSTRAP_KEYS
  ('intelligence:market-implications:v1') and SEED_META
  (seed-meta:intelligence:market-implications, maxStaleMin=150 = 2x
  the 75min TTL, matching gold standard)
- en.json: add components.marketImplications.infoTooltip which was
  referenced in MarketImplicationsPanel but missing from locales

* fix(market-implications): wire CMD+K entry and panels.marketImplications i18n key

- commands.ts: add panel:market-implications command with trade/signal
  keywords so the panel appears in CMD+K search
- en.json: add panels.marketImplications used by UnifiedSettings panel
  toggle display and SearchModal label resolution
2026-03-24 08:01:47 +04:00
Elie Habib
bbffbd6998 feat(economic): BLS direct integration for CES/LAUMT/ECI series (#2141)
* feat(economic): BLS direct integration for CES/LAUMT/ECI series (#2046)

* fix(bls-series): add isMain guard, series allowlist, and empty-value filter

- Wrap runSeed() in isMain guard (process.argv[1]) to prevent seed
  from executing when the module is imported by tests or other scripts.
  This is the critical codebase-wide pattern documented in MEMORY.md.
- Add KNOWN_SERIES_IDS allowlist in getBlsSeries handler to block
  arbitrary Redis key enumeration via user-supplied series_id values.
- Fix observation filter in seed script: add d.value truthiness check
  so empty strings from null/undefined BLS values do not pass through
  as valid observations alongside the existing dash-value guard.

* fix(bls-series): align TTL, cache tier, and maxStaleMin with gold standard

- CACHE_TTL: 86400 → 259200 (3× daily interval)
- maxStaleMin: 1440 → 2880 (2× daily interval)
- gateway cache tier: static → daily (CDN s-maxage=86400 for daily-seeded data)
- process.exit(1) → process.exit(0)

* feat(economic): surface BLS series in EconomicPanel Labor Market tab

Adds Labor Market tab to EconomicPanel (TODO-088) consuming the BLS
direct integration from PR #2141. Closes issue #2046 UI gap.

- fetchBlsData() in economic service: parallel getBlsSeries calls
  for all 5 series, adapted to FredSeries shape with sparkline data
- EconomicPanel: Labor Market tab with national section (payrolls,
  ECI) and Metro Unemployment sub-section (SF, Boston, NYC)
- Tab hidden gracefully when BLS data unavailable (cron not yet run)
- loadBlsData() wired in data-loader parallel task list
- DataSourceId + data-freshness metadata for bls source
- All 21 locales: laborMarket + metroUnemployment keys

* fix(generated): restore main-compatible generated files with BLS additions

* fix(generated): correct indentation from manual merge
2026-03-23 22:40:20 +04:00
Elie Habib
e08457aadf feat(fuel-prices): add retail gasoline and diesel prices panel (#2150)
* feat(fuel-prices): add retail fuel prices panel and seeder

- Add ListFuelPrices proto RPC with FuelPrice/FuelCountryPrice messages
- Create seed-fuel-prices.mjs seeder with 5 sources: Malaysia (data.gov.my),
  Spain (minetur.gob.es), Mexico (datos.gob.mx), US EIA, EU oil bulletin CSV
- Add list-fuel-prices.ts RPC handler reading from Redis seed cache
- Wire handler into EconomicService handler.ts
- Register fuelPrices in cache-keys.ts, bootstrap.js, health.js, gateway.ts
- Add FuelPricesPanel frontend component with gasoline/diesel/source columns
- Wire panel into panel-layout.ts, App.ts (prime + refresh scheduler)
- Add panel config, command entry with fuel/gas/diesel/petrol keywords
- Add fuelPrices refresh interval (6h) in base.ts
- Add i18n keys in en.json (panels.fuelPrices + components.fuelPrices)

Task: fuel-prices

* fix(fuel-prices): add BR/NZ/UK sources, fix EU non-euro currency, review fixes

- Add fetchBrazil() — ANP CSVs (GASOLINA + DIESEL), Promise.allSettled for
  independent partial results, BRL→USD via FX
- Add fetchNewZealand() — MBIE weekly-table.csv, Board price national avg, NZD→USD
- Add fetchUK_ModeA() — CMA retailer JSON feeds (Asda/BP/JET/MFG/Sainsbury's/
  Morrisons), E10+B7 pence→GBP, max-date observedAt across retailers
- Fix EU non-euro members (BG/CZ/DK/HU/PL/RO/SE) using local currency FX on
  EUR-denominated prices — all EU entries now use currency:'EUR'
- Fix fetchBrazil Promise.all → Promise.allSettled (partial CSV failure no
  longer discards both fuels)
- Fix UK observedAt: keep latest date across retailers (not last-processed)
- Fix WoW anomaly: omit wowPct instead of setting to 0
- Lift parseEUPrice out of inner loop to module scope
- Pre-compute parseBRDate per row to avoid double-conversion
- Update infoTooltip: describe methodology without exposing source URLs
- Add BRL, NZD, GBP to FX symbols list

* fix(fuel-prices): fix 4 live data bugs found in external review

EU CSV: replace hardcoded 2024 URLs (both returning 404) with dynamic
discovery — scrape EC energy page for current CSV link, fall back to
generated YYYY-MM patterns for last 4 months.

NZ: live MBIE header has no Region column (Week,Date,Fuel,Variable,Value,
Unit,Status) — remove regionIdx guard that was forcing return []. Values
are in NZD c/L not NZD/L — divide by 100 before storing.

UK: last_updated is DD/MM/YYYY HH:mm:ss not ISO — parse to YYYY-MM-DD
before lexicographic max-date comparison; previous code stored the
seed-run date instead of the latest retailer timestamp.

Panel: source column fell back to — for diesel-only countries because it
only read gas?.source. Use (gas ?? dsl)?.source so diesel-only rows
display their source correctly.
2026-03-23 20:26:57 +04:00
Elie Habib
4f19e36804 feat(intelligence): GDELT tone/vol timeline analysis for escalation signals (#2044) (#2087)
* feat(intelligence): GDELT tone/vol timeline per topic (#2044)

* fix(gdelt-timeline): add isMain guard to seed script, fix gateway cache tier

- Wrap runSeed() call in isMain guard (process.argv[1].endsWith check) to
  prevent CI failures when the seed module is imported rather than executed
  directly — pre-push hook does not catch this
- Change gateway cache tier from 'medium' (20min CDN) to 'daily' (1h
  browser/s-maxage=86400 CDN) to align with the 1h TIMELINE_TTL on the
  per-topic tone/vol Redis keys

* fix(gdelt-timeline): TTL 1h→12h, medium cache tier, real fetchedAt, exit 0

- seed-gdelt-intel.mjs: TIMELINE_TTL 3600→43200 (12h = 2× 6h cron) so
  tone/vol keys survive between cron runs instead of expiring after 1h
- seed-gdelt-intel.mjs: afterPublish wraps tone/vol as {data, fetchedAt}
  so the real write timestamp is stored alongside the arrays
- get-gdelt-topic-timeline.ts: unwrap new envelope shape; fetchedAt now
  reflects actual data write time instead of request time
- gateway.ts: daily→medium cache tier (CDN s-maxage=1200 matches 6h cadence)
- seed-gdelt-intel.mjs: process.exit(1)→0 to match seeder suite convention

* fix(gdelt-timeline): add GdeltTimelinePoint type cast in unwrap helper
2026-03-23 20:10:15 +04:00
Elie Habib
9696a545eb feat(trade): UN Comtrade strategic commodity flows seeder + RPC (#2045) (#2089)
* feat(trade): UN Comtrade strategic commodity flows seeder + RPC (#2045)

* fix(trade): correct byYear overwrite bug and anomaliesOnly upstream flag

Two bugs in the Comtrade flows feature:

1. seed-trade-flows.mjs: byYear Map used year alone as key. With flowCode=X,M
   the API returns both export and import records for the same year; the second
   record silently overwrote the first, causing incorrect val/wt and YoY
   calculations. Fix: key by `${flowCode}:${year}` so exports and imports are
   tracked separately and YoY is computed per flow direction.

2. list-comtrade-flows.ts: `if (!flows.length)` set upstreamUnavailable=true
   even when Redis data was present but all records were filtered out by
   anomaliesOnly=true. Fix: track dataFound separately and only set
   upstreamUnavailable when no Redis keys returned data.

* fix(comtrade-flows): gold standard TTL, maxStaleMin, exit code, batch Redis fetch

- health.js: maxStaleMin 1440→2880 (2× daily interval per gold standard)
- seed-trade-flows.mjs: CACHE_TTL 86400→259200 (72h = 3× daily interval)
- seed-trade-flows.mjs: process.exit(1)→0 to match seeder suite convention
- list-comtrade-flows.ts: replace 30 getCachedJson calls with single getCachedJsonBatch pipeline
2026-03-23 19:52:56 +04:00
Elie Habib
3321069fb3 feat(sanctions): entity lookup index + OpenSanctions search (#2042) (#2085)
* feat(sanctions): entity lookup index + OpenSanctions search (#2042)

* fix: guard tokens[0] access in sanctions lookup

* fix: use createIpRateLimiter pattern in sanctions-entity-search

* fix: add sanctions-entity-search to allowlist and cache tier

* fix: add LookupSanctionEntity RPC to service.proto, regenerate

* fix(sanctions): strip _entityIndex/_state from main key publish, guard limit NaN

P0: seed-sanctions-pressure was writing the full _entityIndex array and _state
snapshot into sanctions:pressure:v1 because afterPublish runs after atomicPublish.
Add publishTransform to strip both fields before the main key write so the
pressure payload stays compact; afterPublish and extraKeys still receive the full
data object and write the correct separate keys.

P1: limit param in sanctions-entity-search edge function passed NaN to OpenSanctions
when a non-numeric value was supplied. Fix with Number.isFinite guard.

P2: add 200-char max length on q param to prevent oversized upstream requests.

* fix(sanctions): maxStaleMin 2x interval, no-store on entity search

health.js: 720min (1x) → 1440min (2x) for both sanctionsPressure and
sanctionsEntities. A single missed 12h cron was immediately flagging stale.

sanctions-entity-search.js: Cache-Control public → no-store. Sanctions
lookups include compliance-sensitive names in the query string; public
caching would have logged/stored these at CDN/proxy layer.
2026-03-23 19:38:11 +04:00
Elie Habib
ddc6603cce feat(infra): Cloudflare Radar DDoS attacks + traffic anomaly endpoints (#2067)
* feat(infra): add Cloudflare Radar DDoS attacks + traffic anomaly endpoints

Extends the existing Cloudflare Radar integration (internet outages) with
two new data streams, both confirmed accessible with the current token:
- L3/L4 DDoS attack summaries (protocol + vector breakdowns, 7d window)
- Traffic anomaly events (DNS/BGP/ICMP anomalies with country + ASN context)

Changes:
- proto: add DdosAttackSummaryEntry + TrafficAnomaly messages; new
  list_internet_ddos_attacks.proto and list_internet_traffic_anomalies.proto;
  wire two new RPCs into InfrastructureService
- buf generate: regenerated server/client TypeScript from updated protos
- seed-internet-outages.mjs: add fetchDdosData() + fetchTrafficAnomalies()
  called inside fetchAll() before runSeed() (process.exit-safe pattern);
  writes cf:radar:ddos:v1 and cf:radar:traffic-anomalies:v1
- list-ddos-attacks.ts + list-traffic-anomalies.ts: read-from-seed handlers
- handler.ts: wire new handlers
- cache-keys.ts + api/bootstrap.js: add ddosAttacks + trafficAnomalies
  bootstrap keys (fast tier); kept in sync to pass bootstrap parity tests
- gateway.ts: add RPC_CACHE_TIER entries (slow) for new routes
- services/infrastructure: add fetchDdosAttacks() + fetchTrafficAnomalies()
  with circuit breakers + hydration support

UI surface (cards alongside outage map) deferred to follow-up.

Closes #2043

* fix(i18n): rename Internet Outages → Internet Disruptions

Broader term covers outages, DDoS events, and traffic anomalies now
seeded from Cloudflare Radar. Updated in en.json (layer label, tooltip,
country brief count strings), map-layer-definitions.ts fallback label,
and commands.ts search keywords.

Other locales retain their translated strings (not degraded — they
already use broader equivalents like "internet disruption" in many langs).

* feat(map): render traffic anomalies + DDoS target locations on disruptions layer

Adds geo-coordinates to both data types so they appear as map markers
under the Internet Disruptions toggle alongside existing outage circles.

- Proto: add latitude/longitude to TrafficAnomaly (fields 10/11), add new
  DdosLocationHit message, add top_target_locations to DdosAttacksResponse
- Seeder: resolve lat/lon from COUNTRY_COORDS for traffic anomalies; fetch
  CF Radar top/locations/target endpoint for DDoS top-target locations
- Server handler: pass topTargetLocations through from Redis seed cache
- DeckGLMap: amber trafficAnomaly layer + purple ddosHit layer with tooltips
- GlobeMap: TrafficAnomalyMarker + DdosHitMarker with emoji indicators
- MapContainer: expose setTrafficAnomalies() + setDdosLocations() setters
- data-loader: fire-and-forget anomaly/DDoS fetches after outages load

* fix(review): address code review findings + add Internet Disruptions panel

- fix: totalCount returns filtered count when country param is set
- fix: countryName uses clientCountryName fallback (was always empty)
- fix: remove duplicate toEpochMsFromIso (consolidate into toEpochMs)
- fix: anomalies guard >= 0 → > 0 (don't write empty array to Redis)
- fix: GlobeMap uses named top-level imports instead of inline imports
- feat: InternetDisruptionsPanel with 3 tabs (Outages / DDoS / Anomalies)
2026-03-22 22:58:41 +04:00
Elie Habib
2dcbf80e4a feat(military): resolve ICAO→IATA callsign for Wingbits flights (#1993)
* feat(military): resolve ICAO→IATA callsign for Wingbits flights

Server resolves ICAO airline prefix (e.g. UAE→EK) at response time using a
static ~100-entry lookup table, populating new proto fields callsign_iata and
airline_name. Schedule lookup falls back to the IATA callsign inside the same
cachedFetchJson callback to recover schedules that ECS only indexes under the
commercial callsign. Callsigns are normalized to uppercase before caching to
prevent duplicate cache keys from mixed-case ECS responses. Client popup shows
the IATA callsign and airline name above the route section.

* fix(military): correct airline data errors and add handler tests

Remove BOG (airport code, not airline ICAO), remove TVS/NOZ (incorrect
Transavia mappings), add TRA→HV as correct Transavia ICAO code. Remove
non-standard TKJ duplicate of THY. Add handler integration tests that
verify the IATA schedule fallback path via globalThis.fetch mock, plus
guards for empty/invalid icao24 inputs.

* docs(military): add refresh comment to airline-codes header

* fix(military): correct TGW and WIF airline mappings

TGW is Scoot (TR), not Oman Air — add OMA for Oman Air (WY).
WIF is Wideroe (WF), not Wizz Air Abu Dhabi — add WAD for Wizz Air Abu Dhabi (W4).
Both errors caused wrong popup labels and broken schedule fallback lookups.

* fix(military): correct AUB and IBK airline mappings

Austrian Airlines uses AUA (not AUB), IATA OS.
Iberia Express uses IBS (not IBK), IATA I2 (not NT).
Both errors caused toIataCallsign() to return null for live traffic,
leaving popup header empty and skipping schedule fallback.

* fix(military): remove/correct five more bad airline ICAO entries

Remove BCS (European Air Transport cargo, not OpenSkies).
Remove SHI (Seoul Air International, no commercial IATA).
Fix SHY: Sky Airlines Turkey (ZY), not Uzbekistan Airways.
Remove WAD (Waddington RAF training unit, not Wizz Air Abu Dhabi).
TGW/Scoot kept as-is — Scoot is the current brand name since 2012.

* fix(military): fix SWG/WestJet mapping, add ASH/Mesa

SWG is Sunwing (WG), not WestJet — add WJA for WestJet (WS).
ASH is Mesa Airlines (YV) — was removed as SHI but correct ICAO never added.
2026-03-21 17:15:29 +04:00
Elie Habib
97f5aa8af7 refactor(proto): extract new Sebuf RPCs from #1399 (lspassos1) (#1888)
* feat(proto): add new RPCs for satellites, oref alerts, telegram, GPS interference, company enrichment

Extracted from #1399 (originally by @lspassos1). Adds 12 new proto message/service
files and updates service.proto for intelligence, aviation, and infrastructure domains.

Intelligence:
- ListSatellites + satellite.proto (TLE orbit data)
- ListOrefAlerts (Israeli Home Front Command alerts)
- ListTelegramFeed (Telegram intelligence feed)
- ListGpsInterference + gps_jamming.proto
- GetCompanyEnrichment (GitHub/SEC/HN enrichment)
- ListCompanySignals

Aviation:
- GetYoutubeLiveStreamInfo

Infrastructure:
- GetBootstrapData
- GetIpGeo
- ReverseGeocode

Co-authored-by: Lucas Passos <lspassos1@users.noreply.github.com>

* chore(proto): regenerate clients/servers/openapi after new RPC additions

* fix(proto): restore GetCountryFacts and ListSecurityAdvisories RPCs removed by contributor

* chore(handlers): add stub implementations for new proto RPCs

* fix(handler): correct stub shapes for GetCompanyEnrichment and ListCompanySignals

* fix(proto): fix handler stub shapes for listOrefAlerts, listTelegramFeed, listGpsInterference

* fix(proto): fix remaining handler stub shapes for aviation and infrastructure

* fix(proto): add cache tier entries for new generated GET routes, remove stale classify-event entry

* fix(pr1888): restore rpc contracts and real handlers

* fix(oref): read history from Redis instead of re-calling relay

relay:oref:history:v1 is seeded by ais-relay on every poll cycle.
History mode now reads directly from Redis (no relay hit).
Live alerts still call relay (in-memory only), with Redis counts as fallback.

* fix(gateway): change youtube-live-stream-info tier from no-store to fast

Matches existing api/youtube/live.js which caches at s-maxage=600.
fast tier = s-maxage=300 stale-while-revalidate=60 — appropriate for
live detection that changes at most every few minutes.

* fix(geocode): await setCachedJson to prevent edge isolate termination race

* fix(youtube): use CHROME_UA constant in fallback fetch paths

* fix(pr1888): address P1/P2 review findings

- gateway: oref+telegram slow->fast (matches legacy s-maxage=300/120)
- gateway: ip-geo slow->no-store (per-request user data, must not share)
- list-gps-interference: recompute stats from filtered hexes when region filter active
- get-company-enrichment: throw ValidationError(400) on missing domain+name
- list-company-signals: throw ValidationError(400) on missing company

* fix(validation): use FieldViolation.description, remove unused buildEmptyResponse

---------

Co-authored-by: Lucas Passos <lspassos1@users.noreply.github.com>
Co-authored-by: lspassos1 <lspassos@icloud.com>
2026-03-21 12:24:59 +04:00
Elie Habib
2e16159bb6 feat(economic): WoW price tracking + weekly cadence for BigMac & Grocery panels (#1974)
* feat(economic): add WoW tracking and fix plumbing for bigmac/grocery-basket panels

Phase 1 — Fix Plumbing:
- Adjust CACHE_TTL to 10 days (864000s) for bigmac and grocery-basket seeds
- Align health.js SEED_META maxStaleMin to 10080 (7 days) for both
- Add grocery-basket and bigmac to seed-health.js SEED_DOMAINS with intervalMin: 5040
- Refactor publish.ts writeSnapshot to accept advanceSeedMeta param; only
  advance seed-meta when fresh data exists (overallFreshnessMin < 120)
- Add manual-fallback-only comment to seed-consumer-prices.mjs

Phase 2 — Week-over-Week Tracking:
- Add wow_pct field to BigMacCountryPrice and CountryBasket proto messages
- Add wow_avg_pct, wow_available, prev_fetched_at to both response protos
- Regenerate client/server TypeScript from updated protos
- Add readCurrentSnapshot() helper + WoW computation to seed-bigmac.mjs
  and seed-grocery-basket.mjs; write :prev key via extraKeys
- Update BigMacPanel.ts to show per-country WoW column and global avg summary
- Update GroceryBasketPanel.ts to show WoW badge on total row and basket avg summary
- Add .bm-wow-up, .bm-wow-down, .bm-wow-summary, .gb-wow CSS classes
- Fix server handlers to include new WoW fields in fallback responses

* fix(economic): guard :prev extraKey against null on first seed run; eliminate double freshness query in publish.ts

* refactor(economic): address code review findings from PR #1974

- Extract readSeedSnapshot() into _seed-utils.mjs (DRY: was duplicated
  verbatim in seed-bigmac and seed-grocery-basket)
- Add FRESH_DATA_THRESHOLD_MIN constant in publish.ts (replace magic 120)
- Fix seed-consumer-prices.mjs contradictory JSDoc (remove stale
  "Deployed as: Railway cron service" line that contradicted manual-only warning)
- Add i18n keys panels.bigmacWow / panels.bigmacCountry to en.json
- Replace hardcoded "WoW" / "Country" with t() calls in BigMacPanel
- Replace IIFE-in-ternary pattern with plain if blocks in BigMacPanel
  and GroceryBasketPanel (P2/P3 from code review)

* fix(publish): gate advanceSeedMeta on any-retailer freshness, not average

overallFreshnessMin is the arithmetic mean across all retailers, so with
1 fresh + 2 stale retailers the average can exceed 120 min and suppress
seed-meta advancement even while fresh data is being published.

Use retailers.some(r => r.freshnessMin < 120) to correctly implement
"at least one retailer scraped within the last 2 hours."
2026-03-21 10:56:48 +04:00
Elie Habib
6dbe4f17bf feat(wingbits): show flight route, times, and plane photo in popup (#1947)
* feat(wingbits): show flight route, times, and plane photo in popup

* chore(gen): regenerate military service stubs after WingbitsLiveFlight proto extension

* fix(wingbits): add empty schedule/photo defaults to mapEcsFlight for type safety

* chore(gen): regenerate MilitaryService OpenAPI docs after WingbitsLiveFlight extension

* fix(wingbits): address code review — sanitizeUrl for img src, DEP/ARR column labels, fmtDelayMin edge case, remove debug artifact
2026-03-20 20:50:35 +04:00