Commit Graph

45 Commits

Author SHA1 Message Date
Elie Habib
636ace7b2c feat(forecast): add impact expansion simulation layer (#2138)
* feat(forecast): add impact expansion simulation layer

* fix(forecast): correct impact bucket coherence gate
2026-03-23 15:19:06 +04:00
Elie Habib
00f9ce7c19 fix(forecast): preserve llm narratives on publish refresh (#2134) 2026-03-23 13:50:57 +04:00
Elie Habib
a202b8ebcc feat(consumer-prices): global All view as default, market selector, per-market cache keys (#2128)
* fix(seeders): apply gold standard TTL-extend+retry pattern to Aviation, NOTAM, Cyber, PositiveEvents

* feat(consumer-prices): default to All — global comparison table as landing view

- DEFAULT_MARKET = 'all' so panel opens with the global view
- 🌍 All pill added at front of market bar
- All view fetches all 9 markets in parallel via fetchAllMarketsOverview()
  and renders a comparison table: Market / Index / WoW / Spread / Updated
- Clicking any market row drills into that market's full tab view
- SINGLE_MARKETS exported for use in All-view iteration
- CSS: .cp-global-table and row styles
2026-03-23 10:58:37 +04:00
Elie Habib
166fc58e92 fix(forecast): tighten state coherence and promotion (#2120)
* fix(forecast): tighten state coherence and promotion

* fix(forecast): harden coherence follow-ups
2026-03-23 10:19:17 +04:00
Elie Habib
1058b648a1 feat(forecast): derive market domains from state units (#2116)
* feat(forecast): derive market domains from state units

* fix(forecast): cover state-derived backfill path
2026-03-23 09:26:13 +04:00
Elie Habib
ea991dc7ce fix(forecast): unclog market promotion and state selection (#2082) 2026-03-23 01:19:28 +04:00
Elie Habib
5e8a106999 feat(forecast): extract critical news signals (#2064)
* feat(forecast): extract critical news signals

* fix(forecast): harden critical signal extraction

* feat(forecast): add structured urgent signal extraction

* docs(env): document critical forecast llm overrides
2026-03-22 22:39:00 +04:00
Elie Habib
a24ea45983 feat(forecast): compress situations into state units (#2037) 2026-03-22 10:11:41 +04:00
Elie Habib
7eef3fd9ca feat(forecast): enrich energy transmission signals (#2021) 2026-03-21 23:05:20 +04:00
Elie Habib
3b762492fe feat(forecast): deepen market transmission simulation (#1996) 2026-03-21 17:16:31 +04:00
Elie Habib
41591e33a9 feat(forecast): add macro market signals (#1980) 2026-03-21 12:26:52 +04:00
Elie Habib
5b987ea434 feat(forecast): drive simulation from market state (#1976) 2026-03-21 11:09:04 +04:00
Elie Habib
3670716daa feat(forecast): add market transmission state (#1971) 2026-03-21 09:48:38 +04:00
Elie Habib
8d86607d21 feat(forecast): drive selection from causal memory (#1958) 2026-03-21 01:20:42 +04:00
Elie Habib
f56f11a596 feat(forecast): add simulation memory replay state (#1945) 2026-03-20 20:37:11 +04:00
Elie Habib
8e8db1b40f feat(forecast): calibrate interaction effect promotion (#1936) 2026-03-20 18:49:58 +04:00
Elie Habib
070248b792 fix(forecast): guarantee military forecast inclusion in publish selection pool (#1917)
Military detector forecasts (ADS-B flight tracking + theater posture API)
structurally score near zero on readiness metrics that require LLM-enriched
caseFile content (supporting evidence, news headlines, calibration, triggers).
This causes them to rank below the target count threshold every run despite
a valid elevated posture signal.

Add a domain guarantee post-pass after the 3 selection loops: if no military
forecast was selected and we have room below MAX_TARGET_PUBLISHED_FORECASTS,
inject the highest-scoring eligible military forecast. This does not displace
any already-selected forecast and respects all existing family/situation caps.

Diagnosis: Baltic theater at postureLevel='elevated' with 6 active flights
generates a military forecast (prob=0.41, confidence=0.30, score=0.136) but
gets buried behind 15+ well-grounded situation cluster forecasts at score 0.4+.

Tests: 3 new assertions in 'military domain guarantee in publish selection'.
2026-03-20 14:04:08 +04:00
Elie Habib
01366fcc00 fix(forecast): block generic-actor cross-theater interactions + raise enrichment budget (#1916)
* fix(forecast): block generic-actor cross-theater interactions + raise enrichment budget

Root cause: actor registry uses name:category as key (e.g. "Incumbent leadership:state"),
causing unrelated situations (Israel conflict, Taiwan political) to share the same actor
ID and fire sharedActor=true in pushInteraction. This propagated into the reportable
ledger and surfaced as junk effects like Israel→Taiwan at 80% confidence.

Two-pronged fix:

1. Specificity gate in pushInteraction: sharedActor now requires avgSpecificity >= 0.75.
   Generic blueprint actors ("Incumbent leadership" ~0.68, "Civil protection authorities"
   ~0.73) no longer qualify as structural cross-situation links. Named domain-specific
   actors ("Threat actors:adversarial" ~0.95) continue to qualify.

2. MACRO_REGION_MAP + isCrossTheaterPair + gate in buildCrossSituationEffects: for
   cross-theater pairs (different macro-regions) with non-exempt channels, requires
   sharedActor=true AND avgActorSpecificity >= 0.90. Exempt channels: cyber_disruption,
   market_repricing (legitimately global). Same-macro-region pairs (Brazil/Mexico both
   AMERICAS) are unaffected.

Verified against live run 1773983083084-bu6b1f:
  BLOCKED: Israel→Taiwan (MENA/EAST_ASIA, spec 0.68)
  BLOCKED: Israel→US political (MENA/AMERICAS, spec 0.68)
  BLOCKED: Cuba→Iran (AMERICAS/MENA, spec 0.73)
  BLOCKED: Brazil→Israel (AMERICAS/MENA, spec 0.85 < 0.90)
  ALLOWED: China→US cyber_disruption (exempt channel)
  ALLOWED: Brazil→Mexico (same AMERICAS)

Also raises ENRICHMENT_COMBINED_MAX from 3 to 5 (total budget 6→8),
targeting enrichedRate improvement from ~38% to ~60%.

* fix(plans): fix markdown lint errors in forecast semantic quality plan

* fix(plans): fix remaining markdown lint error in plan file
2026-03-20 13:26:58 +04:00
Elie Habib
46cd3728d6 fix(forecast): tighten reportable effect quality (#1902)
* fix(forecast): tighten reportable effect quality

* fix(forecast): preserve structural political carryover

* chore(forecast): document effect grouping heuristics
2026-03-20 00:44:21 +04:00
Elie Habib
8768d10b7f fix(forecast): tighten interaction semantics (#1896)
* fix(forecast): tighten interaction semantics

* fix(forecast): narrow maritime family inference

* fix(forecast): keep full reportable interaction graph
2026-03-19 23:34:46 +04:00
Elie Habib
e434769e37 feat(forecast): add simulation action ledger (#1891)
* feat(forecast): add simulation action ledger

* fix(forecast): preserve directional interaction effects
2026-03-19 21:01:47 +04:00
Elie Habib
486f5f799f fix(forecast): tighten family effect credibility (#1880)
* fix(forecast): tighten family effect credibility

* fix(forecast): respect domain effect thresholds
2026-03-19 18:24:40 +04:00
Elie Habib
08cc2723cc fix(forecast): wire per-situation simulation into per-forecast worldState (#1879)
buildForecastTraceArtifacts was building worldState after tracedPredictions,
so simulation data was never available to buildForecastTraceRecord. Each
forecast's caseFile.worldState had situationId/familyId/simulationSummary
all undefined, making the 3-round MiroFish simulation invisible at the
forecast level.

Fix:
- Compute worldState before tracing (so simulationState is ready)
- Build forecastId → situationSimulation lookup from worldState.simulationState
- Pass lookup into buildForecastTraceRecord; inject situationId, familyId,
  familyLabel, simulationSummary, simulationPosture, simulationPostureScore
  into caseFile.worldState for each matched forecast
- Add regression assertion to forecast-trace-export tests

All 194 forecast tests pass.
2026-03-19 17:19:49 +04:00
Elie Habib
2deccac691 fix(forecast): allocate publish output by family (#1868)
* fix(forecast): allocate publish output by family

* fix(forecast): backfill deferred family selections
2026-03-19 11:42:12 +04:00
Elie Habib
ee0f124b3f feat(forecast): add family spillover engine (#1866)
* feat(forecast): add family spillover engine

* fix(forecast): require direct spillover links

* fix(forecast): stabilize family spillover wiring
2026-03-19 10:52:06 +04:00
Elie Habib
0b338afed8 fix(forecast): calibrate simulation posture scoring (#1860)
* fix(forecast): calibrate simulation posture scoring

* fix(forecast): version and rebalance simulation scoring
2026-03-19 09:48:41 +04:00
Elie Habib
15e2a6fccb feat(forecast): drive simulation rounds from actor actions (#1858) 2026-03-19 09:08:04 +04:00
Elie Habib
214b17d757 fix(forecast): align published and candidate state surfaces (#1852)
* fix(forecast): align published and candidate state surfaces

* fix(forecast): preserve projected published situations
2026-03-19 08:24:26 +04:00
Elie Habib
568408b0ca fix(forecast): tighten simulation effect links (#1851) 2026-03-19 03:55:19 +04:00
Elie Habib
10958a397b feat(forecast): synthesize reports from simulation state (#1850) 2026-03-19 03:40:22 +04:00
Elie Habib
a40c0a11fb feat(forecast): add simulation state transitions (#1847) 2026-03-19 03:25:52 +04:00
Elie Habib
a3492e8b4e fix(forecast): refine world-state situation clustering (#1843) 2026-03-19 02:55:22 +04:00
Elie Habib
47e942011b fix(forecast): improve situation-aware publish quality (#1840) 2026-03-19 02:11:27 +04:00
Elie Habib
2b228da916 fix(forecast): dedupe situation-overlap forecasts (#1807)
* fix(forecast): dedupe situation-overlap forecasts

* fix(forecast): reuse situation clusters in publish flow

* fix(forecast): reuse publish trace context
2026-03-18 17:57:03 +04:00
Elie Habib
e58a608262 fix(forecast): make trace writing single-writer (#1801)
* fix(forecast): make trace writing single-writer

* fix(forecast): preserve chained refresh requests
2026-03-18 11:00:43 +04:00
Elie Habib
527002f873 fix(forecast): improve trace enrichment diagnostics (#1797)
* fix(forecast): avoid duplicate prior world-state read

* feat(forecast): record llm enrichment failure reasons

* fix(forecast): preserve latest pointer continuity fallback
2026-03-18 09:55:28 +04:00
Elie Habib
6b1ea49397 feat(forecast): add report continuity history (#1788)
* feat(forecast): cluster situations in world state

* feat(forecast): add report continuity history

* fix(forecast): stabilize report continuity matching
2026-03-18 00:27:03 +04:00
Elie Habib
c1f8aa516b feat(forecast): add situation clustering to world state (#1785)
* feat(forecast): cluster situations in world state

* fix(forecast): stabilize situation continuity ids
2026-03-17 22:47:58 +04:00
Elie Habib
3ba56997af feat(forecast): add world-state report synthesis (#1780) 2026-03-17 19:19:28 +04:00
Elie Habib
0296758398 feat(forecast): add world-state continuity to main (#1779)
* feat(forecast): add actor continuity to world state

* fix(forecast): report full actor continuity counts

* feat(forecast): add branch continuity to world state
2026-03-17 18:49:58 +04:00
Elie Habib
8cdca53bd8 feat(forecast): persist run-level world state (#1773)
* feat(forecast): persist run-level world state

* fix(forecast): align world state artifacts
2026-03-17 18:21:56 +04:00
Elie Habib
e2f0811330 fix(forecast): tighten quality and enrichment balance (#1761) 2026-03-17 14:03:13 +04:00
Elie Habib
ce0f529204 feat(forecast): add trace quality summary (#1746)
* feat(forecast): add trace quality summary

* fix(forecast): split traced and full-run quality metrics
2026-03-17 08:22:20 +04:00
Elie Habib
8619cd16aa fix(forecast): store full traces and tighten matching (#1673)
* fix(forecast): restore missing domains and split cyber

* fix(forecast): store full traces and tighten matching
2026-03-15 19:51:09 +04:00
Elie Habib
39931456a1 feat(forecast): add structured scenario pipeline and trace export (#1646)
* feat(forecast): add AI Forecasts prediction module (Pro-tier)

MiroFish-inspired prediction engine that generates structured forecasts
across 6 domains (conflict, market, supply chain, political, military,
infrastructure) using existing WorldMonitor data streams.

- Proto definitions for ForecastService with GetForecasts RPC
- Dedicated seed script (seed-forecasts.mjs) with 6 domain detectors,
  cross-domain cascade resolver, prediction market calibration, and
  trend detection via prior snapshot comparison
- Premium-gated RPC handler (PREMIUM_RPC_PATHS enforcement)
- Lazy-loaded ForecastPanel with domain filters, probability bars,
  trend arrows, signal evidence, and cascade links
- Health monitoring integration (seed-meta freshness tracking)
- Refresh scheduler with API key guard

* test(forecast): add 47 unit tests for forecast detectors and utilities

Covers forecastId, normalize, resolveCascades, calibrateWithMarkets,
computeTrends, and smoke tests for all 6 domain detectors. Exports
testable functions from seed script with direct-run guard.

* fix(forecast): domain mismatch 'infra' vs 'infrastructure', add panel category

- Seed script used 'infra' but ForecastPanel filtered on 'infrastructure',
  causing Infra tab to show zero results
- Added 'forecast' to intelligence category in PANEL_CATEGORY_MAP

* fix(forecast): move CSS to one-time injection, improve type safety

- P2: Move style block from setContent to one-time document.head injection
  to prevent CSS accumulation on repeated renders
- P3: Replace +toFixed(3) with Math.round for readability in seed script
- P3: Use Forecast type instead of any[] in RPC handler filter

* fix(forecast): handle sebuf proto data shapes from Redis

Detectors now normalize CII scores from server-side proto format
(combinedScore, TREND_DIRECTION_RISING, region) to uniform shape.
Outage severity handles proto enum format (SEVERITY_LEVEL_HIGH).
Added confidence floor of 0.3 for single-source predictions.

Verified against live Redis: 2 predictions generated (Iran infra
shutdown, IL political instability).

* feat(forecast): unlock AI Forecasts on web, lock desktop only (trial)

- Remove forecast RPC from PREMIUM_RPC_PATHS (web access is free)
- Panel locked on desktop only (same as oref-sirens/telegram-intel)
- Remove API key guards from data-loader and refresh scheduler
- Web users get full access during trial period

* chore: regenerate proto types with make generate

Re-ran make generate after rebasing on main. Plugin v0.7.0 dropped
@ts-nocheck from output, added it back to all 50 generated files.
Fixed 4 type errors from proto codegen changes:
- MarketSource enum -> string union type
- TemporalAnomalyProto -> TemporalAnomaly rename
- webcam lastUpdated number -> string

* chore: add proto freshness check to pre-push hook

Runs make generate before push and compares checksums of generated files.
If proto types are stale, blocks push with instructions to regenerate.
Skips gracefully if buf CLI is not installed.

* fix(forecast): use chokepoints v4 key, include ciiContribution in unrest

- P1: Switch chokepoints input from stale v2 to active v4 Redis key,
  matching bootstrap.js and cache-keys.ts
- P2: Add ciiContribution to unrest component fallback chain in
  normalizeCiiEntry so political detector reads the correct sebuf field

* feat(forecast): Phase 2 LLM scenario enrichment + confidence model

MiroFish-inspired enhancements:
- LLM scenario narratives via Groq/OpenRouter (narrative-only, no numeric
  adjustment). Evidence-grounded prompts with mandatory signal citation
  and few-shot examples from MiroFish's SECTION_SYSTEM_PROMPT_TEMPLATE.
- Top-4 predictions batched into single LLM call for cost efficiency.
- News context from newsInsights attached to all predictions for LLM
  prompt grounding (NOT in signals, cannot affect confidence).
- Deterministic confidence model: source diversity via SIGNAL_TO_SOURCE
  mapping (deduplicates cii+cii_delta, theater+indicators) + calibration
  agreement from prediction market drift. Floor 0.2, ceiling 1.0.
- Output validation: rejects scenarios without signal references.
- Truncated JSON repair for small model output.
- Structured JSON logging for LLM calls.
- Redis cache for LLM scenarios (1h TTL).
- 23 new tests (70 total), all passing.
- Live-tested: OpenRouter gemini-2.5-flash produces evidence-grounded
  scenario narratives from real WorldMonitor data.

* feat(forecast): Phase 3 multi-perspective scenarios, projections, data-driven cascades

MiroFish-inspired enhancements:
- Multi-perspective LLM analysis: top-2 predictions get strategic,
  regional, and contrarian viewpoints via combined LLM call
- Probability projections: domain-specific decay curves (h24/d7/d30)
  anchored to timeHorizon so probability equals projections[timeHorizon]
- Data-driven cascade rules: moved from hardcoded array to JSON config
  (scripts/data/cascade-rules.json) with schema validation, named
  predicate evaluators, unknown key rejection, and fallback to defaults
- 4 new cascade paths: infrastructure->supply_chain, infrastructure->market
  (both requiresSeverity:total), conflict->political, political->market
- Proto: added Perspectives and Projections messages to Forecast
- ForecastPanel: renders projections row and conditional perspectives toggle
- 89 tests (19 new), all passing
- Live-tested: OpenRouter produces perspectives from real data

* feat(forecast): Phase 4 data utilization + entity graph

Fixes data gaps that prevented 4 of 6 detectors from firing:
- Input normalizers: chokepoint v4 shape + GPS hexes-to-zones mapping
- Chokepoint warm-ping (production-only, requires WM_API_BASE_URL)
- Lowered CII conflict threshold from 70 to 60, gated on level=high|critical

4 new standalone detectors:
- UCDP conflict zones (10+ events per country)
- Cyber threat concentration (5+ threats per country)
- GPS jamming in maritime shipping zones (5 regions)
- Prediction markets as signals (60-90% probability markets)

Entity-relationship graph (file-based, 38 nodes):
- Countries, theaters, commodities, chokepoints, alliances
- Alias table resolves both ISO codes and display names
- Graph cascade discovery links predictions across entities

Result: 51 predictions (up from 1-2), spanning conflict, infrastructure,
and supply chain domains. 112 tests, all passing.

* fix(forecast): redis cache format, signal source mapping, type safety

Fresh-eyes audit fixes:
- BUG: redisSet used wrong Upstash API format (POST body with {value,ex}
  instead of command array ['SET',key,value,'EX',ttl]). LLM cache writes
  were silently failing, causing fresh LLM calls every run.
- BUG: prediction_market signal type missing from SIGNAL_TO_SOURCE,
  inflating confidence for market-derived predictions.
- CLEANUP: Remove unnecessary (f as any) casts in ForecastPanel since
  generated Forecast type already has projections/perspectives fields.
- CLEANUP: Bump health maxStaleMin from 60 to 90 to avoid false STALE
  alerts when LLM calls add latency to seed runs.

* feat(forecast): headline-entity matching with news corroboration signals

Uses entity graph aliases to match headlines to predictions by
country/theater (excludes commodity/infrastructure nodes to prevent
false positives). Predictions with matching headlines get a
news_corroboration signal visible in the panel.

Also fixes buildUserPrompt to merge unique headlines from ALL
predictions in the LLM batch (was only reading preds[0].newsContext).

Live-tested: 13 of 51 predictions now have corroborating headlines
(Iran, Israel, Syria, Ukraine, etc). 116 tests, all passing.

* feat(forecast): add country-codes.json for headline-entity matching

56 countries with ISO codes, full names, and scoring keywords (extracted
from src/config/countries.ts + UCDP-relevant additions). Used by
attachNewsContext for richer headline matching via getSearchTermsForRegion
which combines country-codes + entity graph + keyword aliases.

14/57 predictions now have news corroboration (limited by headline
coverage, not matching quality: only 8 headlines currently available).

* feat(forecast): read 300 headlines from news digest instead of 8

Read news:digest:v1:full:en (300 headlines across 16 categories) instead
of just news:insights:v1 topStories (8 headlines). Fallback to topStories
if digest is unavailable.

Result: news corroboration jumped from 25% to 64% (38/59 predictions).

* fix(forecast): handle parenthetical country names in headline matching

Strip suffixes like '(Zaire)', '(Burma)', '(Soviet Union)' from UCDP
region names before matching against country-codes.json. Also use
includes() for reverse name lookup to catch partial matches.

Corroboration: 64% -> 69% (41/59). Remaining 18 unmatched are countries
with no current English-language news coverage.

* fix(forecast): cache validated LLM output, add digest test, log cache errors

Fresh-eyes audit fixes:
- Combined LLM cache now stores only validated items (was caching raw
  unvalidated output, serving potentially invalid scenarios on cache hit)
- redisSet logs warnings on failure (was silently swallowing all errors)
- Added digest-based test for attachNewsContext (primary path was untested)
- Fixed test arity: attachNewsContext(preds, news, digest) with 3 params

* fix(forecast): remove dead confidenceFromSources, reduce warm-ping timeout

- P2: Remove confidenceFromSources (dead code, computeConfidence overwrites
  all initial confidence values). Inline the formula in original detectors.
- P3: Reduce warm-ping timeout from 30s to 15s (non-critical step)
- P3: Add trial status comment on forecast panel config

* fix(forecast): resolve ISO codes to country names, fix market detector, safe pre-push

P1 fixes from code review:
- CII ISO codes (IL, IR) now resolved to full country names (Israel, Iran)
  via country-codes.json. Prevents substring false positives (IL matching
  Chile) in event correlation. Uses word-boundary regex for matching.
- Market detector CII-to-theater mapping now uses entity graph traversal
  instead of broken theater-name substring matching. Iran correctly maps
  to Middle East theater via graph links.
- Pre-push hook no longer runs destructive git checkout on proto freshness
  failure. Reports mismatch and exits without modifying worktree.

* feat(forecast): add structured scenario pipeline and trace export

* fix(forecast): hydrate bootstrap and trim generated drift

* fix(forecast): keep required supply-chain contract updates

* fix(ci): add forecasts to cache-keys registry and regenerate proto

Add forecasts entry to BOOTSTRAP_CACHE_KEYS and BOOTSTRAP_TIERS in
cache-keys.ts to match api/bootstrap.js. Regenerate SupplyChain proto
to fix duplicate TransitDayCount and add riskSummary/riskReportAction.
2026-03-15 15:57:22 +04:00