mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-26 01:24:59 +02:00
45f5e5a457
* feat(forecast): add AI Forecasts prediction module (Pro-tier)
MiroFish-inspired prediction engine that generates structured forecasts
across 6 domains (conflict, market, supply chain, political, military,
infrastructure) using existing WorldMonitor data streams.
- Proto definitions for ForecastService with GetForecasts RPC
- Dedicated seed script (seed-forecasts.mjs) with 6 domain detectors,
cross-domain cascade resolver, prediction market calibration, and
trend detection via prior snapshot comparison
- Premium-gated RPC handler (PREMIUM_RPC_PATHS enforcement)
- Lazy-loaded ForecastPanel with domain filters, probability bars,
trend arrows, signal evidence, and cascade links
- Health monitoring integration (seed-meta freshness tracking)
- Refresh scheduler with API key guard
* test(forecast): add 47 unit tests for forecast detectors and utilities
Covers forecastId, normalize, resolveCascades, calibrateWithMarkets,
computeTrends, and smoke tests for all 6 domain detectors. Exports
testable functions from seed script with direct-run guard.
* fix(forecast): domain mismatch 'infra' vs 'infrastructure', add panel category
- Seed script used 'infra' but ForecastPanel filtered on 'infrastructure',
causing Infra tab to show zero results
- Added 'forecast' to intelligence category in PANEL_CATEGORY_MAP
* fix(forecast): move CSS to one-time injection, improve type safety
- P2: Move style block from setContent to one-time document.head injection
to prevent CSS accumulation on repeated renders
- P3: Replace +toFixed(3) with Math.round for readability in seed script
- P3: Use Forecast type instead of any[] in RPC handler filter
* fix(forecast): handle sebuf proto data shapes from Redis
Detectors now normalize CII scores from server-side proto format
(combinedScore, TREND_DIRECTION_RISING, region) to uniform shape.
Outage severity handles proto enum format (SEVERITY_LEVEL_HIGH).
Added confidence floor of 0.3 for single-source predictions.
Verified against live Redis: 2 predictions generated (Iran infra
shutdown, IL political instability).
* feat(forecast): unlock AI Forecasts on web, lock desktop only (trial)
- Remove forecast RPC from PREMIUM_RPC_PATHS (web access is free)
- Panel locked on desktop only (same as oref-sirens/telegram-intel)
- Remove API key guards from data-loader and refresh scheduler
- Web users get full access during trial period
* chore: regenerate proto types with make generate
Re-ran make generate after rebasing on main. Plugin v0.7.0 dropped
@ts-nocheck from output, added it back to all 50 generated files.
Fixed 4 type errors from proto codegen changes:
- MarketSource enum -> string union type
- TemporalAnomalyProto -> TemporalAnomaly rename
- webcam lastUpdated number -> string
* fix(forecast): use chokepoints v4 key, include ciiContribution in unrest
- P1: Switch chokepoints input from stale v2 to active v4 Redis key,
matching bootstrap.js and cache-keys.ts
- P2: Add ciiContribution to unrest component fallback chain in
normalizeCiiEntry so political detector reads the correct sebuf field
* feat(forecast): Phase 2 LLM scenario enrichment + confidence model
MiroFish-inspired enhancements:
- LLM scenario narratives via Groq/OpenRouter (narrative-only, no numeric
adjustment). Evidence-grounded prompts with mandatory signal citation
and few-shot examples from MiroFish's SECTION_SYSTEM_PROMPT_TEMPLATE.
- Top-4 predictions batched into single LLM call for cost efficiency.
- News context from newsInsights attached to all predictions for LLM
prompt grounding (NOT in signals, cannot affect confidence).
- Deterministic confidence model: source diversity via SIGNAL_TO_SOURCE
mapping (deduplicates cii+cii_delta, theater+indicators) + calibration
agreement from prediction market drift. Floor 0.2, ceiling 1.0.
- Output validation: rejects scenarios without signal references.
- Truncated JSON repair for small model output.
- Structured JSON logging for LLM calls.
- Redis cache for LLM scenarios (1h TTL).
- 23 new tests (70 total), all passing.
- Live-tested: OpenRouter gemini-2.5-flash produces evidence-grounded
scenario narratives from real WorldMonitor data.
* feat(forecast): Phase 3 multi-perspective scenarios, projections, data-driven cascades
MiroFish-inspired enhancements:
- Multi-perspective LLM analysis: top-2 predictions get strategic,
regional, and contrarian viewpoints via combined LLM call
- Probability projections: domain-specific decay curves (h24/d7/d30)
anchored to timeHorizon so probability equals projections[timeHorizon]
- Data-driven cascade rules: moved from hardcoded array to JSON config
(scripts/data/cascade-rules.json) with schema validation, named
predicate evaluators, unknown key rejection, and fallback to defaults
- 4 new cascade paths: infrastructure->supply_chain, infrastructure->market
(both requiresSeverity:total), conflict->political, political->market
- Proto: added Perspectives and Projections messages to Forecast
- ForecastPanel: renders projections row and conditional perspectives toggle
- 89 tests (19 new), all passing
- Live-tested: OpenRouter produces perspectives from real data
* feat(forecast): Phase 4 data utilization + entity graph
Fixes data gaps that prevented 4 of 6 detectors from firing:
- Input normalizers: chokepoint v4 shape + GPS hexes-to-zones mapping
- Chokepoint warm-ping (production-only, requires WM_API_BASE_URL)
- Lowered CII conflict threshold from 70 to 60, gated on level=high|critical
4 new standalone detectors:
- UCDP conflict zones (10+ events per country)
- Cyber threat concentration (5+ threats per country)
- GPS jamming in maritime shipping zones (5 regions)
- Prediction markets as signals (60-90% probability markets)
Entity-relationship graph (file-based, 38 nodes):
- Countries, theaters, commodities, chokepoints, alliances
- Alias table resolves both ISO codes and display names
- Graph cascade discovery links predictions across entities
Result: 51 predictions (up from 1-2), spanning conflict, infrastructure,
and supply chain domains. 112 tests, all passing.
* fix(forecast): redis cache format, signal source mapping, type safety
Fresh-eyes audit fixes:
- BUG: redisSet used wrong Upstash API format (POST body with {value,ex}
instead of command array ['SET',key,value,'EX',ttl]). LLM cache writes
were silently failing, causing fresh LLM calls every run.
- BUG: prediction_market signal type missing from SIGNAL_TO_SOURCE,
inflating confidence for market-derived predictions.
- CLEANUP: Remove unnecessary (f as any) casts in ForecastPanel since
generated Forecast type already has projections/perspectives fields.
- CLEANUP: Bump health maxStaleMin from 60 to 90 to avoid false STALE
alerts when LLM calls add latency to seed runs.
* feat(forecast): headline-entity matching with news corroboration signals
Uses entity graph aliases to match headlines to predictions by
country/theater (excludes commodity/infrastructure nodes to prevent
false positives). Predictions with matching headlines get a
news_corroboration signal visible in the panel.
Also fixes buildUserPrompt to merge unique headlines from ALL
predictions in the LLM batch (was only reading preds[0].newsContext).
Live-tested: 13 of 51 predictions now have corroborating headlines
(Iran, Israel, Syria, Ukraine, etc). 116 tests, all passing.
* feat(forecast): add country-codes.json for headline-entity matching
56 countries with ISO codes, full names, and scoring keywords (extracted
from src/config/countries.ts + UCDP-relevant additions). Used by
attachNewsContext for richer headline matching via getSearchTermsForRegion
which combines country-codes + entity graph + keyword aliases.
14/57 predictions now have news corroboration (limited by headline
coverage, not matching quality: only 8 headlines currently available).
* feat(forecast): read 300 headlines from news digest instead of 8
Read news:digest:v1:full:en (300 headlines across 16 categories) instead
of just news:insights:v1 topStories (8 headlines). Fallback to topStories
if digest is unavailable.
Result: news corroboration jumped from 25% to 64% (38/59 predictions).
* fix(forecast): handle parenthetical country names in headline matching
Strip suffixes like '(Zaire)', '(Burma)', '(Soviet Union)' from UCDP
region names before matching against country-codes.json. Also use
includes() for reverse name lookup to catch partial matches.
Corroboration: 64% -> 69% (41/59). Remaining 18 unmatched are countries
with no current English-language news coverage.
* fix(forecast): cache validated LLM output, add digest test, log cache errors
Fresh-eyes audit fixes:
- Combined LLM cache now stores only validated items (was caching raw
unvalidated output, serving potentially invalid scenarios on cache hit)
- redisSet logs warnings on failure (was silently swallowing all errors)
- Added digest-based test for attachNewsContext (primary path was untested)
- Fixed test arity: attachNewsContext(preds, news, digest) with 3 params
* fix(forecast): remove dead confidenceFromSources, reduce warm-ping timeout
- P2: Remove confidenceFromSources (dead code, computeConfidence overwrites
all initial confidence values). Inline the formula in original detectors.
- P3: Reduce warm-ping timeout from 30s to 15s (non-critical step)
- P3: Add trial status comment on forecast panel config
* fix(forecast): resolve ISO codes to country names, fix market detector, safe pre-push
P1 fixes from code review:
- CII ISO codes (IL, IR) now resolved to full country names (Israel, Iran)
via country-codes.json. Prevents substring false positives (IL matching
Chile) in event correlation. Uses word-boundary regex for matching.
- Market detector CII-to-theater mapping now uses entity graph traversal
instead of broken theater-name substring matching. Iran correctly maps
to Middle East theater via graph links.
- Pre-push hook no longer runs destructive git checkout on proto freshness
failure. Reports mismatch and exits without modifying worktree.
1 line
15 KiB
JSON
1 line
15 KiB
JSON
{"components":{"schemas":{"AcledConflictEvent":{"description":"AcledConflictEvent represents an armed conflict event from the ACLED dataset.","properties":{"actors":{"items":{"description":"Named actors involved in the event.","type":"string"},"type":"array"},"admin1":{"description":"Administrative region within the country.","type":"string"},"country":{"description":"Country where the event occurred.","type":"string"},"eventType":{"description":"ACLED event type classification (e.g., \"Battles\", \"Explosions/Remote violence\").","type":"string"},"fatalities":{"description":"Reported fatalities from this event.","format":"int32","type":"integer"},"id":{"description":"Unique ACLED event identifier.","minLength":1,"type":"string"},"location":{"$ref":"#/components/schemas/GeoCoordinates"},"occurredAt":{"description":"Time the event occurred, as Unix epoch milliseconds.. Warning: Values \u003e 2^53 may lose precision in JavaScript","format":"int64","type":"integer"},"source":{"description":"Source article or report.","type":"string"}},"required":["id"],"type":"object"},"Error":{"description":"Error is returned when a handler encounters an error. It contains a simple error message that the developer can customize.","properties":{"message":{"description":"Error message (e.g., 'user not found', 'database connection failed')","type":"string"}},"type":"object"},"FieldViolation":{"description":"FieldViolation describes a single validation error for a specific field.","properties":{"description":{"description":"Human-readable description of the validation violation (e.g., 'must be a valid email address', 'required field missing')","type":"string"},"field":{"description":"The field path that failed validation (e.g., 'user.email' for nested fields). For header validation, this will be the header name (e.g., 'X-API-Key')","type":"string"}},"required":["field","description"],"type":"object"},"GeoCoordinates":{"description":"GeoCoordinates represents a geographic location using WGS84 coordinates.","properties":{"latitude":{"description":"Latitude in decimal degrees (-90 to 90).","format":"double","maximum":90,"minimum":-90,"type":"number"},"longitude":{"description":"Longitude in decimal degrees (-180 to 180).","format":"double","maximum":180,"minimum":-180,"type":"number"}},"type":"object"},"GetHumanitarianSummaryBatchRequest":{"description":"GetHumanitarianSummaryBatchRequest looks up humanitarian summaries for multiple countries.","properties":{"countryCodes":{"items":{"description":"ISO 3166-1 alpha-2 country codes (e.g., \"YE\", \"SD\"). Max 25.","maxItems":25,"minItems":1,"type":"string"},"maxItems":25,"minItems":1,"type":"array"}},"type":"object"},"GetHumanitarianSummaryBatchResponse":{"description":"GetHumanitarianSummaryBatchResponse contains humanitarian summaries for the requested countries.","properties":{"fetched":{"description":"Number of countries successfully fetched.","format":"int32","type":"integer"},"requested":{"description":"Number of countries requested.","format":"int32","type":"integer"},"results":{"additionalProperties":{"$ref":"#/components/schemas/HumanitarianCountrySummary"},"description":"Map of country_code -\u003e humanitarian summary for found countries.","type":"object"}},"type":"object"},"GetHumanitarianSummaryRequest":{"description":"GetHumanitarianSummaryRequest specifies which country to retrieve the humanitarian summary for.","properties":{"countryCode":{"description":"ISO 3166-1 alpha-2 country code (e.g., \"YE\", \"SD\", \"SO\").","pattern":"^[A-Z]{2}$","type":"string"}},"required":["countryCode"],"type":"object"},"GetHumanitarianSummaryResponse":{"description":"GetHumanitarianSummaryResponse contains the humanitarian summary for the requested country.","properties":{"summary":{"$ref":"#/components/schemas/HumanitarianCountrySummary"}},"type":"object"},"HumanitarianCountrySummary":{"description":"HumanitarianCountrySummary represents HAPI conflict event counts for a country.","properties":{"conflictDemonstrations":{"description":"Number of demonstration events.","format":"int32","type":"integer"},"conflictEventsTotal":{"description":"Total conflict events in the reference period.","format":"int32","type":"integer"},"conflictFatalities":{"description":"Total fatalities from political violence and civilian targeting.","format":"int32","type":"integer"},"conflictPoliticalViolenceEvents":{"description":"Political violence + civilian targeting event count.","format":"int32","type":"integer"},"countryCode":{"description":"ISO 3166-1 alpha-2 country code.","type":"string"},"countryName":{"description":"Country name.","type":"string"},"referencePeriod":{"description":"Reference period start date (YYYY-MM-DD).","type":"string"},"updatedAt":{"description":"Last data update time, as Unix epoch milliseconds.. Warning: Values \u003e 2^53 may lose precision in JavaScript","format":"int64","type":"integer"}},"type":"object"},"IranEvent":{"properties":{"category":{"type":"string"},"id":{"type":"string"},"latitude":{"format":"double","type":"number"},"locationName":{"type":"string"},"longitude":{"format":"double","type":"number"},"severity":{"type":"string"},"sourceUrl":{"type":"string"},"timestamp":{"format":"int64","type":"string"},"title":{"type":"string"}},"type":"object"},"ListAcledEventsRequest":{"description":"ListAcledEventsRequest specifies filters for retrieving ACLED conflict events.","properties":{"country":{"description":"Optional country filter (ISO 3166-1 alpha-2).","type":"string"},"cursor":{"description":"Cursor for next page.","type":"string"},"end":{"description":"End of time range (inclusive), Unix epoch milliseconds.. Warning: Values \u003e 2^53 may lose precision in JavaScript","format":"int64","type":"integer"},"pageSize":{"description":"Maximum items per page (1-100).","format":"int32","type":"integer"},"start":{"description":"Start of time range (inclusive), Unix epoch milliseconds.. Warning: Values \u003e 2^53 may lose precision in JavaScript","format":"int64","type":"integer"}},"type":"object"},"ListAcledEventsResponse":{"description":"ListAcledEventsResponse contains ACLED conflict events matching the request.","properties":{"events":{"items":{"$ref":"#/components/schemas/AcledConflictEvent"},"type":"array"},"pagination":{"$ref":"#/components/schemas/PaginationResponse"}},"type":"object"},"ListIranEventsRequest":{"type":"object"},"ListIranEventsResponse":{"properties":{"events":{"items":{"$ref":"#/components/schemas/IranEvent"},"type":"array"},"scrapedAt":{"format":"int64","type":"string"}},"type":"object"},"ListUcdpEventsRequest":{"description":"ListUcdpEventsRequest specifies filters for retrieving UCDP violence events.","properties":{"country":{"description":"Optional country filter (ISO 3166-1 alpha-2).","type":"string"},"cursor":{"description":"Cursor for next page.","type":"string"},"end":{"description":"End of time range (inclusive), Unix epoch milliseconds.. Warning: Values \u003e 2^53 may lose precision in JavaScript","format":"int64","type":"integer"},"pageSize":{"description":"Maximum items per page (1-100).","format":"int32","type":"integer"},"start":{"description":"Start of time range (inclusive), Unix epoch milliseconds.. Warning: Values \u003e 2^53 may lose precision in JavaScript","format":"int64","type":"integer"}},"type":"object"},"ListUcdpEventsResponse":{"description":"ListUcdpEventsResponse contains UCDP violence events matching the request.","properties":{"events":{"items":{"$ref":"#/components/schemas/UcdpViolenceEvent"},"type":"array"},"pagination":{"$ref":"#/components/schemas/PaginationResponse"}},"type":"object"},"PaginationResponse":{"description":"PaginationResponse contains pagination metadata returned alongside list results.","properties":{"nextCursor":{"description":"Cursor for fetching the next page. Empty string indicates no more pages.","type":"string"},"totalCount":{"description":"Total count of items matching the query, if known. Zero if the total is unknown.","format":"int32","type":"integer"}},"type":"object"},"ResultsEntry":{"properties":{"key":{"type":"string"},"value":{"$ref":"#/components/schemas/HumanitarianCountrySummary"}},"type":"object"},"UcdpViolenceEvent":{"description":"UcdpViolenceEvent represents a georeferenced violence event from the UCDP dataset.","properties":{"country":{"description":"Country where the event occurred.","type":"string"},"dateEnd":{"description":"End date of the event, as Unix epoch milliseconds.. Warning: Values \u003e 2^53 may lose precision in JavaScript","format":"int64","type":"integer"},"dateStart":{"description":"Start date of the event, as Unix epoch milliseconds.. Warning: Values \u003e 2^53 may lose precision in JavaScript","format":"int64","type":"integer"},"deathsBest":{"description":"Best estimate of deaths.","format":"int32","type":"integer"},"deathsHigh":{"description":"High estimate of deaths.","format":"int32","type":"integer"},"deathsLow":{"description":"Low estimate of deaths.","format":"int32","type":"integer"},"id":{"description":"Unique UCDP event identifier.","minLength":1,"type":"string"},"location":{"$ref":"#/components/schemas/GeoCoordinates"},"sideA":{"description":"Primary party in the conflict (Side A).","type":"string"},"sideB":{"description":"Secondary party in the conflict (Side B).","type":"string"},"sourceOriginal":{"description":"Original source of the event report.","type":"string"},"violenceType":{"description":"UcdpViolenceType represents the UCDP violence classification.\n Maps to existing TS union: 'state-based' | 'non-state' | 'one-sided'.","enum":["UCDP_VIOLENCE_TYPE_UNSPECIFIED","UCDP_VIOLENCE_TYPE_STATE_BASED","UCDP_VIOLENCE_TYPE_NON_STATE","UCDP_VIOLENCE_TYPE_ONE_SIDED"],"type":"string"}},"required":["id"],"type":"object"},"ValidationError":{"description":"ValidationError is returned when request validation fails. It contains a list of field violations describing what went wrong.","properties":{"violations":{"description":"List of validation violations","items":{"$ref":"#/components/schemas/FieldViolation"},"type":"array"}},"required":["violations"],"type":"object"}}},"info":{"title":"ConflictService API","version":"1.0.0"},"openapi":"3.1.0","paths":{"/api/conflict/v1/get-humanitarian-summary":{"get":{"description":"GetHumanitarianSummary retrieves a humanitarian overview for a country from HAPI/HDX.","operationId":"GetHumanitarianSummary","parameters":[{"description":"ISO 3166-1 alpha-2 country code (e.g., \"YE\", \"SD\", \"SO\").","in":"query","name":"country_code","required":false,"schema":{"type":"string"}}],"responses":{"200":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/GetHumanitarianSummaryResponse"}}},"description":"Successful response"},"400":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ValidationError"}}},"description":"Validation error"},"default":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}},"description":"Error response"}},"summary":"GetHumanitarianSummary","tags":["ConflictService"]}},"/api/conflict/v1/get-humanitarian-summary-batch":{"post":{"description":"GetHumanitarianSummaryBatch retrieves humanitarian summaries for multiple countries in one call.","operationId":"GetHumanitarianSummaryBatch","requestBody":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/GetHumanitarianSummaryBatchRequest"}}},"required":true},"responses":{"200":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/GetHumanitarianSummaryBatchResponse"}}},"description":"Successful response"},"400":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ValidationError"}}},"description":"Validation error"},"default":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}},"description":"Error response"}},"summary":"GetHumanitarianSummaryBatch","tags":["ConflictService"]}},"/api/conflict/v1/list-acled-events":{"get":{"description":"ListAcledEvents retrieves armed conflict events from the ACLED dataset.","operationId":"ListAcledEvents","parameters":[{"description":"Start of time range (inclusive), Unix epoch milliseconds.","in":"query","name":"start","required":false,"schema":{"format":"int64","type":"string"}},{"description":"End of time range (inclusive), Unix epoch milliseconds.","in":"query","name":"end","required":false,"schema":{"format":"int64","type":"string"}},{"description":"Maximum items per page (1-100).","in":"query","name":"page_size","required":false,"schema":{"format":"int32","type":"integer"}},{"description":"Cursor for next page.","in":"query","name":"cursor","required":false,"schema":{"type":"string"}},{"description":"Optional country filter (ISO 3166-1 alpha-2).","in":"query","name":"country","required":false,"schema":{"type":"string"}}],"responses":{"200":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ListAcledEventsResponse"}}},"description":"Successful response"},"400":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ValidationError"}}},"description":"Validation error"},"default":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}},"description":"Error response"}},"summary":"ListAcledEvents","tags":["ConflictService"]}},"/api/conflict/v1/list-iran-events":{"get":{"description":"ListIranEvents retrieves scraped conflict events from LiveUAMap Iran.","operationId":"ListIranEvents","responses":{"200":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ListIranEventsResponse"}}},"description":"Successful response"},"400":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ValidationError"}}},"description":"Validation error"},"default":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}},"description":"Error response"}},"summary":"ListIranEvents","tags":["ConflictService"]}},"/api/conflict/v1/list-ucdp-events":{"get":{"description":"ListUcdpEvents retrieves georeferenced violence events from the UCDP dataset.","operationId":"ListUcdpEvents","parameters":[{"description":"Start of time range (inclusive), Unix epoch milliseconds.","in":"query","name":"start","required":false,"schema":{"format":"int64","type":"string"}},{"description":"End of time range (inclusive), Unix epoch milliseconds.","in":"query","name":"end","required":false,"schema":{"format":"int64","type":"string"}},{"description":"Maximum items per page (1-100).","in":"query","name":"page_size","required":false,"schema":{"format":"int32","type":"integer"}},{"description":"Cursor for next page.","in":"query","name":"cursor","required":false,"schema":{"type":"string"}},{"description":"Optional country filter (ISO 3166-1 alpha-2).","in":"query","name":"country","required":false,"schema":{"type":"string"}}],"responses":{"200":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ListUcdpEventsResponse"}}},"description":"Successful response"},"400":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ValidationError"}}},"description":"Validation error"},"default":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}},"description":"Error response"}},"summary":"ListUcdpEvents","tags":["ConflictService"]}}}} |