mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-05-13 02:26:22 +02:00
9b94bbc625fdf4e158519903e1c9d6b9de10def3
119 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Elie Habib
|
0cdfddc885 |
feat(gold): central-bank reserves via IMF IFS (PR C) (#3038)
* feat(gold): central-bank gold reserves via IMF IFS (PR C) * fix(gold): prefer ounces indicator over USD in IMF IFS candidate list * fix(gold): align seed-health interval with monthly IMF cadence + drop ALG dup Review findings on PR #3038: - api/seed-health.js: intervalMin was 1440 (1 day), which flags stale at 2880min (48h) — contradicted health.js maxStaleMin=44640 (~31 days) and would false-alarm within 2 days on a monthly data source. Bumped to 22320 so both endpoints agree at ~31 days. - seed-gold-cb-reserves ISO3_NAMES: dropped duplicate ALG entry (World Bank variant); DZA is canonical ISO 3166-1 alpha-3 and stays. |
||
|
Elie Habib
|
a8b85e52c8 |
feat(gold): SPDR GLD physical holdings flows (PR B) (#3037)
* feat(gold): SPDR GLD physical holdings flows (PR B) * fix(gold): strip UTF-8 BOM from SPDR CSV header (greptile P2 #3037) |
||
|
Elie Habib
|
ee66b6b5c2 |
feat(gold): Gold Intelligence v2 — positioning depth, returns, drivers (#3034)
* feat(gold): richer Gold Intelligence panel with positioning, returns, drivers * fix(gold): restore leveragedFunds fields and derive P/S netPct in legacy fallback Review catch on PR #3034: 1. seed-cot.mjs stopped emitting leveragedFundsLong/Short during the v2 refactor, which would zero out the Leveraged Funds bars in the existing CotPositioningPanel on the next seed run. Re-read lev_money_* from the TFF rows and keep the fields on the output (commodity rows don't have this breakdown, stay at 0). 2. get-gold-intelligence legacy fallback hardcoded producerSwap.netPct to 0, meaning a pre-v2 COT snapshot rendered a neutral 0% Producer/Swap bar on deploy until seed-cot reran. Derive netPct from dealerLong/dealerShort (same formula as the v2 seeder). OI share stays 0 because open_interest wasn't captured pre-migration; clearly documented now. Tests: added two regression guards (leveragedFunds preserved for TFF, commodity rows emit 0 for those fields). * fix(gold): make enrichment layer monitored and honest about freshness Review catch on PR #3034: - seed-commodity-quotes now writes seed-meta:market:gold-extended via writeExtraKeyWithMeta on every successful run. Partial / failed fetches skip BOTH the data write and the meta bump, so health correctly reports STALE_SEED instead of masking a broken Yahoo fetch with a green check. - Require both gold (core) AND at least one driver/silver before writing, so a half-successful run doesn't overwrite healthy prior data with a degraded payload. - Handler no longer stamps updatedAt with new Date() when the enrichment key is missing. Emits empty string so the panel's freshness indicator shows "Updated —" with a dim dot, matching reality — enrichment is missing, not fresh. - Health: goldExtended entry in STANDALONE_KEYS + SEED_META (maxStaleMin 30, matching commodity quotes), and seed-health.js advertises the domain so upstream monitors pick it up. The panel already gates session/returns/drivers sections on presence, so legacy panels without the enrichment layer stay fully functional. |
||
|
Elie Habib
|
281a7c0728 | chore: regenerate MarketService OpenAPI specs for GetGoldIntelligence (#3011) | ||
|
Elie Habib
|
793d7df9dc | feat(energy-crisis): add IEA 2026 Energy Crisis Policy Response Tracker panel and seeder (#3008) | ||
|
Elie Habib
|
c26ae6b827 | feat(energy): add Oil Inventories panel with SVG charts (#3003) | ||
|
Elie Habib
|
c72251178c |
feat(route-explorer): Sprint 4 — strategic-product impact tab + get-route-impact RPC (#2996)
* feat(route-explorer): Sprint 4 — strategic-product impact tab
Adds the Impact tab to the Route Explorer, powered by a new
get-route-impact RPC that returns strategic-product trade data for
any country pair.
Backend:
- New proto get_route_impact.proto with GetRouteImpact{Request,Response}
+ StrategicProduct message
- New handler server/worldmonitor/supply-chain/v1/get-route-impact.ts:
reads comtrade:bilateral-hs4:{iso2}:v1 store, computes lane value for
selected HS2, top 5 strategic products by value with chokepoint
exposure, resilience score (server-side from Redis), dependency flags
- Cache key ROUTE_IMPACT_KEY in cache-keys.ts (NOT in BOOTSTRAP_KEYS)
- Gateway + premium-paths registered as slow-browser premium RPC
- Client wrapper fetchRouteImpact in supply-chain/index.ts
Impact tab UI:
- CountryImpactTab.ts: strategic products table (top 5 by value),
lane value card for selected HS2, hs2InSeededUniverse banner when
HS2 is not in the 14 seeded sectors, comtradeSource states
(missing/empty/bilateral-hs4), drill-sideways on product row click
- LeftRail.updateDependencyFlags: renders flags from Impact response
with color-coded badges (compound_risk/single_source/diversifiable)
Data flow:
- fetchImpact fires in parallel with fetchResilience after lane data
loads, generation-scoped
- Impact response updates left-rail flags + resilience score
- Drill-sideways: clicking a product row switches the explorer's HS2
and re-queries all tabs
Server-side resilience:
- get-route-impact reads resilience:score:v8:{iso2} from Redis directly
so the data is available for future email briefs without client calls
Plan: docs/plans/2026-04-11-001-feat-worldwide-route-explorer-plan.md
* fix(route-explorer): real exposure score for flags + tabstrip sync on drill
P1: computeDependencyFlags hardcoded primaryExposure=80 whenever any
chokepoint existed, fabricating SINGLE_CORRIDOR_CRITICAL without using
real exposure data. Replaced with computeRealExposureScore that uses the
same route-cluster overlap logic as get-sector-dependency, computing the
actual exposure percentage before comparing against the >80 threshold.
P2: handleDrillSideways set state.tab=1 directly without going through
setTab(), leaving the tabstrip visually and semantically on Impact while
content showed Current. Now calls setTab(1) which updates both the
tabstrip active state and aria-selected.
* fix(route-explorer): guard resilience overwrite + normalize HS2 filter
P1: fetchImpact could zero the left-rail resilience score when
get-route-impact returned resilienceScore=0 (Redis miss fallback),
overwriting a valid score set by the concurrent fetchResilience call.
Now only applies the server-side score when it is actually > 0.
P2: HS4-to-HS2 matching used a redundant dual-condition filter
(hs4ToHs2 + startsWith) that masked a potential normalization bug.
Simplified to normalize hs2 once via parseInt then use a single
hs4ToHs2 comparison.
|
||
|
Elie Habib
|
e070a97c3d |
Phase 3 PR2: Weekly regional briefs (LLM seeder + RPC) (#2989)
* feat(intelligence): weekly regional briefs (Phase 3 PR2) Phase 3 PR2 of the Regional Intelligence Model. Adds LLM-powered weekly intelligence briefs per region, completing the core feature set. ## New seeder: scripts/seed-regional-briefs.mjs Standalone weekly cron script (not part of the 6h derived-signals bundle). For each non-global region: 1. Read the latest snapshot via two-hop Redis read 2. Read recent regime transitions from the history log (#2981) 3. Call the LLM once per region with regime trajectory + balance + triggers + narrative context 4. Write structured brief to intelligence:regional-briefs:v1:weekly:{region} with 8-day TTL (survives one missed weekly run) Reuses the same injectable-callLlm + parse-validation + provider-chain pattern from narrative.mjs and weekly-brief.mjs. ## New module: scripts/regional-snapshot/weekly-brief.mjs generateWeeklyBrief(region, snapshot, transitions, opts?) -> { region_id, generated_at, period_start, period_end, situation_recap, regime_trajectory, key_developments[], risk_outlook, provider, model } buildBriefPrompt() — pure prompt builder parseBriefJson() — JSON parser with prose-extraction fallback emptyBrief() — canonical empty shape Global region is skipped. Provider chain: Groq -> OpenRouter. Validate callback ensures only parseable responses pass (narrative.mjs PR #2960 review fix pattern). ## Proto + RPC: GetRegionalBrief proto/worldmonitor/intelligence/v1/get_regional_brief.proto - GetRegionalBriefRequest { region_id } - GetRegionalBriefResponse { brief: RegionalBrief } - RegionalBrief { region_id, generated_at, period_start, period_end, situation_recap, regime_trajectory, key_developments[], risk_outlook, provider, model } ## Server handler server/worldmonitor/intelligence/v1/get-regional-brief.ts Simple getCachedJson read + adaptBrief snake->camel adapter. Returns upstreamUnavailable: true on Redis failure so the gateway skips caching (matching the get-regime-history pattern from #2981). ## Premium gating + cache tier src/shared/premium-paths.ts + server/gateway.ts RPC_CACHE_TIER ## Tests — 27 new unit tests buildBriefPrompt (5): region/balance/transitions/narrative rendered, empty transitions handled, missing fields tolerated parseBriefJson (5): valid JSON, garbage, all-empty, cap at 5, prose extraction generateWeeklyBrief (6): success, global skip, LLM fail, garbage, exception, period_start/end delta emptyBrief (2): region_id + empty fields handler (4): key prefix, adapter export, upstreamUnavailable, registration security (2): premium path + cache tier proto (3): RPC declared, import wired, RegionalBrief fields ## Verification - npm run test:data: 4651/4651 pass - npm run typecheck + typecheck:api: clean - biome lint: clean * fix(intelligence): address 3 review findings on #2989 P2 #1 — no consumer surface for GetRegionalBrief Acknowledged. The consumer is the RegionalIntelligenceBoard panel, which will call GetRegionalBrief and render a weekly brief block. This wiring is Phase 3 PR3 (UI) scope — the RPC + Redis key are the delivery mechanism, not the end surface. No code change in this commit; the RPC is ready for the panel to consume. P2 #2 — readRecentTransitions collapses failure to [] readRecentTransitions returned [] on Redis/network failure, which is indistinguishable from a genuinely quiet week. The LLM then generates a brief claiming "no regime transitions" when in reality the upstream is down — fabricating false input. Fix: return null on failure. The seeder skips the region with a clear log message when transitions is null, so the brief is never written with unreliable input. Empty array [] now only means genuinely no transitions in the 7-day window. P2 #3 — parseBriefJson accepts briefs the seeder rejects parseBriefJson treated non-empty key_developments as valid even if situation_recap was empty. The seeder gate only writes when brief.situation_recap is truthy. That mismatch means the validator pass + provider-fallback logic could accept a response that the seeder then silently drops. Fix: require situation_recap in parseBriefJson for valid=true, matching the seeder gate. Now both checks agree on what constitutes a usable brief, and the provider-fallback chain correctly falls through when a provider returns a brief with developments but no recap. * fix(intelligence): TTL path-segment fix + seed-meta always-write (Greptile P1+P2 on #2989) P1 — TTL silently not applied (briefs never expire) Upstash REST ignores query-string SET options (?EX=N). The correct form is path-segment: /set/{key}/{value}/EX/{seconds}. Without this fix every brief persists indefinitely and Redis storage grows unboundedly across weekly runs. P2 — seed-meta not written when all regions skipped writeExtraKeyWithMeta was gated on generated > 0. If every region was skipped (no snapshot yet, or LLM failed), seed-meta was never written, making the seeder indistinguishable from "never ran" in health tooling. Now writes seed-meta whenever failed === 0, carrying regionsSkipped count. P2 #3 (validate gate) — already fixed in previous commit (parseBriefJson now requires situation_recap for valid=true). * fix(intelligence): register regional-briefs in health.js SEED_META + STANDALONE_KEYS (review P2 on #2989) * fix(intelligence): register regional-briefs in api/seed-health.js (review P2 on #2989) * fix(intelligence): raise brief TTL to 15 days to cover missed weekly cycle (review P2 on #2989) * fix(intelligence): distinguish missing-key from Redis-error + coverage-gated health (review P2s on #2989) P2 #1 — false upstreamUnavailable before first seed getCachedJson returns null for both "key missing" and "Redis failed", so the handler was advertising an outage for every region before the first weekly seed ran. Switched to getRawJson (throws on Redis errors) so null = genuinely missing key → clean empty 200, and thrown error = upstream failure → upstreamUnavailable: true for gateway no-store. P2 #2 — partial run hides coverage loss in health The seed-meta was written with generated count even if only 1 of 7 regions produced a brief. /api/health treats any positive recordCount as healthy, so broad regional failure was invisible to operators. Fix: recordCount is set to 0 when generated < ceil(expectedRegions/2). This makes /api/health report EMPTY_DATA for severely partial runs while still writing seed-meta (so the seeder is confirmed to have run). coverageOk flag in the summary payload lets operators drill into the exact coverage state. * fix(intelligence): tighten coverage gate to expectedRegions-1 (review P2 on #2989) |
||
|
Elie Habib
|
39d5199ae0 |
feat(resilience): three-pillar schema + schemaVersion v2.0 feature flag (Phase 2 T2.1) (#2977)
* feat(resilience): three-pillar schema + schemaVersion v2.0 feature flag (Phase 2 T2.1) Ships the Phase 2 T2.1 schema slice of the country-resilience reference grade upgrade plan. Adds the three-pillar response shape (StructuralReadiness, LiveShockExposure, RecoveryCapacity) as a new `pillars` field on GetResilienceScoreResponse alongside a `schemaVersion` string field, both gated behind the RESILIENCE_SCHEMA_V2_ENABLED env flag (default false). This PR is schema + plumbing only. Pillars ship with score=0, coverage=0; real aggregation lands in PR 4 (T2.3). No behavior change at the v1 default, which preserves widget / map / Country Brief compatibility for one release cycle per the plan. What this PR commits - Proto: new ResiliencePillar message (id, score, weight, coverage, domains). New `pillars` repeated field + `schema_version` string field on GetResilienceScoreResponse. No renumbering or mutation of existing fields. - Generated TS: regenerated service_server.ts, service_client.ts, and OpenAPI JSON/YAML via `make generate`. - New module server/worldmonitor/resilience/v1/_pillar-membership.ts: declarative PILLAR_DOMAINS map + PILLAR_WEIGHTS map + ordered iteration list. Single source of truth for the pillar structure that PR 4 will import. Note: pillar membership uses the runtime ResilienceDomainId values (kebab-case domain ids that already ship in v1), not the long-form pillar names from the plan example. - structural-readiness (0.40): economic, infrastructure, social-governance - live-shock-exposure (0.35): energy, health-food - recovery-capacity (0.25): empty until PR 3 adds the new dimensions - Response builder: new buildPillarList helper emits shaped-but-empty pillars when the v2 flag is on, empty array when off. Response literal fallback paths in _shared.ts and the LOCKED_PREVIEW fixture in resilience-widget-utils.ts updated to include pillars: [] and schemaVersion: '1.0' to satisfy the generated TS types. - Tests: 13 new pillar-schema unit cases (membership invariants, weight sum=1.0, disjoint sets, empty recovery pillar, buildPillarList flag-off / flag-on / shuffled-order / partial-domain-set) + 3 response-shape cases on the release-gate test pinning the v1 default shape and the new field presence on the wire. What is deliberately NOT in this PR - No aggregation logic: score/coverage on pillars stay 0 until PR 4. - No cache key bump: schema is additive with proto3 defaults. - No changes to overallScore/baselineScore/stressScore (parallel for one release cycle). - No new seeders or dimensions (PR 3 / T2.2b). - No tiering registry changes (PR 2 / T2.2a). - No widget rendering (Phase 3 T3.6). Verified - make generate clean, new ResiliencePillar interface in regenerated client + server + OpenAPI artifacts - typecheck + typecheck:api clean - tests/resilience-pillar-schema.test.mts: 13/13 passing - tests/resilience-release-gate.test.mts: 14/14 passing (3 new T2.1 cases + 11 prior) - full resilience suite: 283/283 passing - npm run test:data: 4539/4539 passing - npm run lint: exit 0 * fix(resilience): cache-flag decoupling + freshness error-status guard (#2977 P1+P2) Two Greptile review findings addressed: P1: RESILIENCE_SCHEMA_V2_ENABLED changed the cached response shape but the cache key did not encode the flag state. Flipping the flag on a warm cache served stale v1.0 payloads until the 6h TTL expired. Fix: always compute and cache the v2 superset (with pillars and schemaVersion='2.0'). Apply the flag as a response-time gate: when off, strip pillars to [] and downgrade schemaVersion to '1.0' before returning. This decouples the cache from the flag and makes flag flips take effect immediately without waiting for TTL expiry. P2: readFreshnessMap in _dimension-freshness.ts trusted fetchedAt without checking status. The resilience-static seeder writes fetchedAt: Date.now() on BOTH success and error paths (status: 'ok' vs 'error'), so a failed seed run that preserved old data via extendExistingTtl made the freshness badges show 'fresh' for what is actually stale data. Fix: skip seed-meta entries where status !== 'ok'. When the meta is skipped, the dimension has no freshness data and classifies as stale, matching api/health.js behavior. Added a test case that verifies error-status entries are excluded from the freshness map. |
||
|
Elie Habib
|
822eef0fa6 |
feat(supply-chain): Sprint 1 — Route Explorer wrapper RPC (#2980)
* feat(supply-chain): Sprint 1 — Route Explorer wrapper RPC
Adds an internal wrapper around the vendor-only route-intelligence
compute so the upcoming Route Explorer UI can call it from a browser
PRO session instead of forcing an X-WorldMonitor-Key API gate.
Backend:
- New proto get-route-explorer-lane.proto with GetRouteExplorerLane{Request,Response}
- New handler server/worldmonitor/supply-chain/v1/get-route-explorer-lane.ts
- New static lookup tables _route-explorer-static-tables.ts:
TRANSIT_DAYS_BY_ROUTE_ID, FREIGHT_USD_BY_CARGO_TYPE,
BYPASS_CORRIDOR_GEOMETRY_BY_ID — covers all 5 land-bridge corridors
plus every sea-alternative corridor with hand-curated coordinates
- Wired into supply-chain handler.ts service dispatcher
- Cache key ROUTE_EXPLORER_LANE_KEY in cache-keys.ts (NOT in BOOTSTRAP_KEYS)
- Gateway entry: PREMIUM_RPC_PATHS + RPC_CACHE_TIER 'slow-browser'
- Premium path entry in src/shared/premium-paths.ts so browser PRO auth attaches
Response contract enriches route-intelligence with:
- primaryRouteGeometry polyline from TRADE_ROUTES (lon/lat pairs)
- fromPort/toPort coords on every bypass option so the client can call
MapContainer.setBypassRoutes directly without geometry lookups
- status: 'active' | 'proposed' | 'unavailable' derived from corridor notes
to honestly label kra_canal_future and black_sea_western_ports
- estTransitDaysRange + estFreightUsdPerTeuRange from static tables
- noModeledLane: true when origin/destination clusters share no routes
Client wrapper fetchRouteExplorerLane added to src/services/supply-chain/index.ts.
Tests: tests/route-explorer-lane.test.mts — 30-query smoke matrix
(10 country pairs × 3 HS2 codes), structural assertions only, no
hard-coded transit/cost values. Test exposes a pure computeLane()
function with an injectable status map so it does not need Redis.
Gap report (from smoke run): 12 of 30 queries fall back to a synthetic
primaryRouteId because the destination's port cluster has no shared route
with the origin (US-JP, ZA-IN, CL-CN, TR-DE × 3 HS2 each). These pairs
return noModeledLane:true; Sprint 3 will render an empty-state for them.
Plan: docs/plans/2026-04-11-001-feat-worldwide-route-explorer-plan.md
* fix(route-explorer): address PR #2980 review findings
P1: bypass warRiskTier was hard-coded to WAR_RISK_TIER_NORMAL, dropping
the live risk signal from chokepoint status. Now derived from the
statusMap via the corridor's primaryChokepointId.
P2: freight fallback in emptyResponse and client-side empty payload used
a cargo-agnostic container range for all cargo types. Removed the ranges
entirely from fallback/noModeledLane responses; they are only present
when the lane is actually modeled.
Suggestion: when noModeledLane is true, the response now returns empty
primaryRouteId, empty geometry, empty exposures, empty bypasses, and
omits transit/freight ranges. Previously it returned plausible-looking
synthetic data from the origin's first route which could mislead the UI.
Tests updated to assert the noModeledLane contract: empty fields when
the flag is set, non-empty ranges only when the lane is modeled.
* fix(route-explorer): cargo-aware route ranking + bypass waypoint risk
P1: primary route selection was order-dependent, picking whichever
shared route the origin cluster listed first. Mixed clusters like
CN/JP could return an energy lane for a container request. Now ranks
shared routes by cargo-category compatibility (container→container,
tanker→energy, bulk→bulk, roro→container) before selecting.
P1: bypass warRiskTier was copied from the primary chokepoint instead
of derived from the corridor's own waypointChokepointIds. This
overstated risk for alternatives like Cape of Good Hope whose waypoints
may have a lower risk tier. Now uses max-tier across waypoint
chokepoints, matching get-bypass-options.ts logic.
Suggestion: placeholder corridors with addedTransitDays=0 (like
gibraltar_no_bypass, cape_of_good_hope_is_bypass) are now filtered out.
Previously they could surface as active alternatives.
Regression tests added:
- CN→JP tanker: asserts energy route is selected over container route
- CN→DE with faked Suez=CRITICAL / Cape=NORMAL: asserts Cape bypass
shows NORMAL, not CRITICAL
- ES→EG: asserts zero-transit-day placeholders are excluded
* fix(route-explorer): scope exposures to primary route + narrow placeholder filter
P1: chokepointExposures and bypassOptions were computed from the full
sharedRoutes set, mixing data from energy/container corridors into a
single response. Now scoped to the cargo-ranked primaryRouteId only,
matching the proto contract that exposures are "on the primary route."
P2: the addedTransitDays === 0 filter was too broad and removed
kra_canal_future (a proposed bypass with real modeling). Narrowed to an
explicit PLACEHOLDER_CORRIDOR_IDS set (gibraltar_no_bypass,
cape_of_good_hope_is_bypass) so proposed zero-day corridors survive and
are surfaced with CORRIDOR_STATUS_PROPOSED.
Regression tests:
- chokepointExposures follow primaryRouteId (CN->JP container)
- kra_canal_future appears as CORRIDOR_STATUS_PROPOSED for Malacca routes
- placeholder filter still excludes explicit placeholders
* fix(route-explorer): address PR #2980 review comments
1. Unavailable corridors without waypoints (e.g. black_sea_western_ports)
now derive WAR_RISK_TIER_WAR_ZONE from their CORRIDOR_STATUS_UNAVAILABLE
status, instead of returning WAR_RISK_TIER_UNSPECIFIED. Corridors with
waypointChokepointIds still use max-tier across those waypoints.
2. Added fixture test with non-empty status map (suez=75/HIGH,
malacca=30/ELEVATED) so disruptionScore and warRiskTier assertions are
not trivially satisfied by the empty-map default path.
3. Documented the single-chokepoint bypass design gap in the test gap report:
bypassOptions only cover the primary chokepoint; multi-chokepoint routes
show exposure for all but bypass guidance for only the top one. Sprint 3
will decide whether to expand to top-N or add a UI hint.
|
||
|
Elie Habib
|
19d67cea94 |
Phase 3 PR1: Regime drift history (writer + RPC) (#2981)
* feat(intelligence): regime drift history (Phase 3 PR1)
Phase 3 PR1 of the Regional Intelligence Model. Adds an append-only
regime transition log per region plus a premium-gated RPC to read it.
## What landed
### New writer module: scripts/regional-snapshot/regime-history.mjs
Single public entry point:
recordRegimeTransition(region, snapshot, diff, opts?)
-> { recorded, entry, pushed, trimmed }
Pure builder + Redis-ops orchestrator + dependency-injected publisher.
Flow:
1. buildTransitionEntry() returns null when diff has no regime_changed
(steady-state snapshots produce no entry — pure transition stream)
2. publishTransitionWithOps() LPUSHes onto
intelligence:regime-history:v1:{region}, then LTRIMs to keep the
most recent REGIME_HISTORY_MAX (100) entries
3. defaultPublisher binds real Upstash REST calls; tests inject an
in-memory ops object for offline coverage
LTRIM failure is non-fatal — entry already landed, next cycle will
re-trim. LPUSH failure short-circuits and reports pushed=false. The
recorder NEVER throws and is wrapped in its own try/catch in the seed
loop so snapshot persist is never blocked.
### seed-regional-snapshots.mjs hook
Added a regime-history call alongside the existing alert-emitter call,
right after persistSnapshot success. Same best-effort contract:
unconditional try/catch, log warn on throw, continue main loop.
### Proto + RPC: GetRegimeHistory
proto/worldmonitor/intelligence/v1/get_regime_history.proto
- GetRegimeHistoryRequest { region_id, limit (0..100) }
- GetRegimeHistoryResponse { transitions: RegimeTransition[] }
- RegimeTransition { region_id, label, previous_label,
transitioned_at, transition_driver, snapshot_id }
region_id validated as strict kebab-case (same regex as
get-regional-snapshot). limit capped server-side at MAX_LIMIT=100,
defaulting to 50 when omitted.
Added to IntelligenceService in service.proto. Generated openapi
JSON/YAML committed via `make generate`.
### Server handler: server/worldmonitor/intelligence/v1/get-regime-history.ts
LRANGE-based read (newest-first because the writer LPUSHes). adapter
is a dedicated exported function adaptTransition(raw) for testability.
LRANGE helper is inlined here because server/_shared/redis.ts has no
list helpers yet — this is the first list-reading handler in the
intelligence service. If a second list reader lands, the helper can
be promoted to a shared util.
Empty list / Redis miss / failed JSON parse all return
{ transitions: [] } so the client can distinguish "never changed" from
"upstream broken" via the HTTP status code, not the body.
Registered in handler.ts.
### Premium gating + cache tier
src/shared/premium-paths.ts: added /api/intelligence/v1/get-regime-history
server/gateway.ts RPC_CACHE_TIER: same path with 'slow' tier (matches
route-parity contract enforced by
tests/route-cache-tier.test.mjs)
## Tests — 44 new unit tests
tests/regional-snapshot-regime-history.test.mjs (22 tests):
buildTransitionEntry (7):
- null on missing diff/region/snapshot
- returns entry on regime change
- first-ever transition (empty previous_label)
- falls back to generated_at when transitioned_at is missing
- preserves snapshot_id
publishTransitionWithOps (8):
- happy path (LPUSH + LTRIM both succeed)
- canonical key prefix
- LTRIM uses REGIME_HISTORY_MAX-1 stop
- LPUSH failure → not pushed, LTRIM not called
- LTRIM failure → pushed=true, trimmed=false (non-fatal)
- LPUSH/LTRIM throwing caught and reported
- null/empty entry → no-op
recordRegimeTransition (5):
- no-op on no regime change
- records on regime change
- publisher returning false → recorded=false
- publisher exceptions swallowed
- critical escalation labels preserved
module constants (2): key prefix + max are valid
tests/get-regime-history.test.mts (22 tests):
adaptTransition (4):
- all fields snake → camel
- missing fields → empty/zero defaults
- first-ever transition shape preserved
- non-numeric transitioned_at → 0
handler structural checks (7): canonical key prefix, LRANGE usage,
adapter export, handler export signature, MAX_LIMIT cap matches
writer, missing-region short-circuit, malformed-entry filter
intelligence handler registration (2): import + registration
security wiring (2): premium path + cache-tier entry
proto definition (7): RPC method declared, import wired, request
shape, kebab regex, limit bounds, RegimeTransition fields,
response shape
## Verification
- node --test tests/regional-snapshot-regime-history.test.mjs: 22/22 pass
- npx tsx --test tests/get-regime-history.test.mts: 22/22 pass
- npm run test:data: 4621/4621 pass
- npm run typecheck: clean
- npm run typecheck:api: clean
- biome lint on touched files: clean
## Deferred to future iterations
- Phase 3 PR2: weekly regional briefs LLM seeder (consumes regime history
to highlight drift events in the weekly summary)
- Phase 3 PR3: UI block in RegionalIntelligenceBoard for regime drift
timeline (can ride alongside or after PR2)
- Drift analytics: % of last N days spent in each regime, transition
frequency rolling window, regime cycle detection
- Alert triggers on drift cycles (e.g., "thrashed between regimes 3 times
in 7 days")
* fix(intelligence): address 2 review findings on #2981
P2 #1 — transition_driver always empty in the live path
buildRegimeState(balance, previousLabel, '') at Step 11 passed an empty
driver because the diff hasn't been computed yet. The regime-history
recorder reads snapshot.regime.transition_driver which was therefore
always '' in production, despite tests exercising synthetic fixtures
with a populated driver.
Fix: after Step 15 derives triggerReason via inferTriggerReason(diff),
backfill regime.transition_driver = triggerReason when a genuine regime
change occurred. This ensures both the persisted snapshot's regime block
AND the regime-history entry carry the real driver (e.g., 'regime_shift',
'trigger_activation', 'corridor_break').
Added 2 regression tests: populated driver flows through, and pre-fix
empty-driver snapshots remain back-compatible.
P2 #2 — Redis failure returns cached false-empty history
get-regime-history.ts returned 200 {transitions:[]} on LRANGE failure.
The gateway caches 200 GET responses at the slow tier, so a transient
Upstash outage would be pinned as a false-empty history until the cache
TTL expired.
Fix: when redisLrange returns null (Redis unavailable or credentials
missing), the response now includes upstreamUnavailable: true in the
body. The gateway already checks for this flag in the response body
(line 434) and sets Cache-Control: no-store, so transient failures are
not cached.
Added 1 structural test asserting the upstreamUnavailable flag is set.
Verification:
- 24/24 writer tests, 23/23 handler tests, 4624/4624 full suite pass
- npm run typecheck: clean
- biome lint on touched files: clean
* fix(intelligence): correct misleading 'log once per region' comment (Greptile P2)
|
||
|
Elie Habib
|
75b3b0026b |
feat(resilience): dimension freshness propagation (T1.5 propagation pass) (#2961)
* feat(resilience): dimension freshness propagation (T1.5 propagation pass) Ships the Phase 1 T1.5 propagation pass of the country-resilience reference-grade upgrade plan. PR #2947 shipped the staleness classifier foundation (classifyStaleness, cadence taxonomy, three staleness levels) and explicitly deferred the dimension-level propagation. This PR consumes the classifier and surfaces per dimension freshness on the ResilienceDimension response. What this PR commits - Proto: new DimensionFreshness message + `freshness` field on ResilienceDimension (last_observed_at_ms, staleness string). - New module server/worldmonitor/resilience/v1/_dimension-freshness.ts that reads seed-meta values for every sourceKey in INDICATOR_REGISTRY and aggregates the worst staleness + oldest fetchedAt across the constituent indicators of each dimension. - scoreAllDimensions decorates each dimension score with its freshness result before returning. The 13 dimension scorer function bodies are untouched: aggregation is a decoration pass at the caller level so this PR stays mechanical. - Response builder: _shared.ts buildDimensionList propagates the freshness field to the proto output. - Tests: 10 classifyDimensionFreshness + readFreshnessMap cases in a new test file + response-shape case on the release-gate test. Aggregation rules - last_observed_at_ms: MIN fetchedAt across the dimension's indicators (oldest signal = most conservative bound). 0 when no signal has ever been observed. - staleness: MAX staleness level across the dimension's indicators (stale > aging > fresh). Empty string when the dimension has no indicators in the registry (defensive path). What is deliberately NOT in this PR - No changes to the 13 individual dimension scorer function bodies. Per-signal freshness inside scorers is a future enhancement. - No widget rendering of the freshness badge (T1.6 full grid, PR 3). - No cache key bump: additive int64/string fields with zero defaults. Verified - make generate clean, new interface in regenerated types - typecheck + typecheck:api clean - tests/resilience-dimension-freshness.test.mts all new cases pass - tests/resilience-*.test.mts full suite pass - test:data clean - lint exits 0 on touched files * fix(resilience): resolve templated sourceKeys to real seed-meta (#2961 P1) Greptile P1 finding on PR #2961: readFreshnessMap() assumed every INDICATOR_REGISTRY sourceKey could be fetched as seed-meta:<sourceKey>, but most entries use placeholder templates like resilience:static:{ISO2}, energy:mix:v1:{ISO2}, and displacement:summary:v1:{year}. Those produce literal lookups like seed-meta:resilience:static:{ISO2} which don't exist in Redis, so the freshness map missed every templated entry and classifyDimensionFreshness marked the affected dimensions stale even with healthy seeds. Most Phase 1 T1.5 freshness badges were broken on arrival. Fix: two-layer resolution in _dimension-freshness.ts. Layer 1 stripTemplateTokens: drop :{placeholder} and :* segments. 'resilience:static:{ISO2}' -> 'resilience:static' 'resilience:static:*' -> 'resilience:static' 'energy:mix:v1:{ISO2}' -> 'energy:mix:v1' 'displacement:summary:v1:{year}' -> 'displacement:summary:v1' Layer 2 stripTrailingVersion: strip trailing :v\d+, mirroring writeExtraKeyWithMeta + runSeed() in scripts/_seed-utils.mjs which never persist the trailing version in seed-meta keys. Handles cyber:threats:v2, infra:outages:v1, unrest:events:v1, conflict:ucdp-events:v1, sanctions:country-counts:v1, and the displacement v1 case above. Layer 3 SOURCE_KEY_META_OVERRIDES: explicit table for drift cases where the two strips still do not match the real seed-meta key. Verified against api/seed-health.js, api/health.js, and scripts/seed-*. Drift cases covered: economic:imf:macro -> economic:imf-macro economic:bis:eer -> economic:bis economic:energy:v1:all -> economic:energy-prices energy:mix -> economic:owid-energy-mix energy:gas-storage -> energy:gas-storage-countries news:threat:summary -> news:threat-summary intelligence:social:reddit -> intelligence:social-reddit readFreshnessMap now deduplicates reads by resolved meta key (so the 15+ resilience:static indicators share one Redis read) and projects per-meta-key results back onto per-sourceKey map entries so classifyDimensionFreshness can keep its existing interface. Regression coverage: - stripTemplateTokens cases for {ISO2}, {year}, and *. - stripTrailingVersion cases for :v1 / :v2 suffixes. - Embedded :v1 carve-out (trade:restrictions:v1:tariff-overview:50 stays unchanged because :v1 is not trailing). - Override cases for the seven drift entries. - Integration test that proves every resilience:static:* / {ISO2} registry entry resolves to the same seed-meta and is marked fresh when that one key has a recent fetchedAt. - healthPublicService end-to-end test: classifies fresh when seed-meta:resilience:static is recent (was stale before the fix). - Registry-coverage assertion: every INDICATOR_REGISTRY sourceKey must resolve to a seed-meta key that either lives in api/seed-health.js, api/health.js, or the test's KNOWN_SEEDS_NOT_IN_HEALTH allowlist (which covers the four seeds written by writeExtraKeyWithMeta / runSeed that no health monitor tracks yet: trade:restrictions, trade:barriers, sanctions:country-counts, economic:energy-prices). Fails loudly if a future registry entry introduces an unknown sourceKey. Note on P1 #2 (scoreCurrencyExternal absence-branch delete): that is PR #2964's scope (T1.7 source-failure wiring), not #2961 (T1.5 propagation pass). #2961 never claimed to delete the fallback branch; no test in this branch expects the new IMPUTE.bisEer fallback. The reviewer conflated the two stacked PRs. #2964 owns the delete. |
||
|
Elie Habib
|
dca2e1ca3c |
feat(resilience): expose imputationClass on ResilienceDimension (T1.7 schema pass) (#2959)
* feat(resilience): expose imputationClass on ResilienceDimension (T1.7 schema pass) Ships the Phase 1 T1.7 schema pass of the country-resilience reference grade upgrade plan. PR #2944 shipped the classifier table foundation (ImputationClass type, ImputationEntry interface, IMPUTATION/IMPUTE tagged with four semantic classes) and explicitly deferred the schema propagation. This PR lands that propagation so downstream consumers can distinguish "country is stable" from "country is unmonitored" from "upstream is down" from "structurally not applicable" on a per-dimension basis. What this PR commits - Proto: new imputation_class string field on ResilienceDimension (empty string = dimension has any observed data; otherwise one of stable-absence, unmonitored, source-failure, not-applicable). - Generated TS types: regenerated service_server.ts and service_client.ts via make generate. - Scorer: ResilienceDimensionScore carries ImputationClass | null. WeightedMetric carries an optional imputationClass that imputation paths populate. weightedBlend aggregates the dominant class by weight when the dimension is fully imputed, returns null otherwise. - All IMPUTE.* early-return paths propagate the class from the table (IMPUTE.bisEer, IMPUTE.wtoData, IMPUTE.ipcFood, IMPUTE.unhcrDisplacement). - Response builder: _shared.ts buildDimensionList passes the class through to the ResilienceDimension proto field. - Tests: weightedBlend aggregation semantics (5 cases), dimension-level propagation from IMPUTE tables, serialized response includes the field. What is deliberately NOT in this PR - No widget icon rendering (T1.6 full grid, PR 3 of 5) - No source-failure seed-meta consultation (PR 4 of 5) - No freshness field (T1.5 propagation, PR 2 of 5) - No cache key bump: the new field is empty-string default, existing cached responses continue to deserialize cleanly Verified - make generate clean - npm run typecheck + typecheck:api clean - tests/resilience-dimension-scorers.test.mts all passing (existing + new) - tests/resilience-*.test.mts + test:data suite passing (4361 tests) - npm run lint exits 0 * fix(resilience): normalize cached score responses on read (#2959 P2) Greptile P2 finding on PR #2959: cachedFetchJson and getCachedResilienceScores return pre-change payloads verbatim, so a resilience:score:v7 entry written before this PR lands lacks the imputationClass field. Downstream consumers that read dim.imputationClass get undefined for up to 6 hours until the cache TTL expires. Fix: add normalizeResilienceScoreResponse helper that defaults missing optional fields in place and apply it at both read sites. Defaults imputationClass to empty string, matching the proto3 default for the new imputation_class field. - ensureResilienceScoreCached applies the normalizer after cachedFetchJson returns. - getCachedResilienceScores applies it after each successful JSON.parse on the pipeline result. - Two new test cases: stale payload without imputationClass gets defaulted, present values are preserved. - Not bumping the cache key: stale-read defaults are safe, the key bump would invalidate every cached score for a 6-hour cold-start cycle. The normalizer is extensible when PR #2961 adds freshness to the same payload. P3 finding (broken docs reference) verified invalid: the proto comment points to docs/methodology/country-resilience-index.mdx, which IS the current file. The .md predecessor was renamed in PR #2945 (T1.3 methodology doc promotion to CII parity). No change needed to the comment. * fix(resilience): bump score cache key v7 to v8, drop normalizer (#2959 P2) Second fixup for the Greptile P2 finding on #2959. The previous fixup ( |
||
|
Elie Habib
|
7da202c25d |
Phase 1 PR1: RegionalSnapshot proto + RPC handler (#2951)
* feat(intelligence): add RegionalSnapshot proto definition
Defines the canonical RegionalSnapshot wire format for Phase 1 of the
Regional Intelligence Model. Mirrors the TypeScript contract in
shared/regions.types.d.ts that Phase 0 landed with.
New proto file: proto/worldmonitor/intelligence/v1/get_regional_snapshot.proto
Messages:
- RegionalSnapshot (13 top-level fields matching the spec)
- SnapshotMeta (11 fields including snapshot_id, narrative_provider,
narrative_model, trigger_reason, snapshot_confidence, missing_inputs,
stale_inputs, valid_until, versions)
- RegimeState (label + transition history)
- BalanceVector (7 axes: 4 pressures + 3 buffers + net_balance + decomposed
drivers)
- BalanceDriver (axis, magnitude, evidence_ids, orientation)
- ActorState (leverage_score, role, domains, delta, evidence_ids)
- LeverageEdge (actor-to-actor directed influence)
- ScenarioSet + ScenarioLane (per-horizon distribution normalizing to 1.0)
- TransmissionPath (typed fields: severity, confidence, latency_hours,
magnitude range, asset class, template provenance)
- TriggerLadder + Trigger + TriggerThreshold (structured operator/value/
window/baseline)
- MobilityState + AirspaceStatus + FlightCorridorStress + AirportNodeStatus
- EvidenceItem (typed origin for the trust trail)
- RegionalNarrative + NarrativeSection (LLM-synthesized text with
evidence_ids on every section)
RPC: GetRegionalSnapshot(GetRegionalSnapshotRequest) -> GetRegionalSnapshotResponse
- GET /api/intelligence/v1/get-regional-snapshot
- region_id validated as lowercase kebab via buf.validate regex
- No other parameters; the handler reads canonical state
Generated code committed alongside:
- src/generated/client/worldmonitor/intelligence/v1/service_client.ts
- src/generated/server/worldmonitor/intelligence/v1/service_server.ts
- docs/api/IntelligenceService.openapi.{json,yaml}
The generated TypeScript types use camelCase per standard buf codegen, while
Phase 0 persists snapshots in Redis using the snake_case shape from
shared/regions.types.d.ts. The handler lands in a follow-up commit with a
localized snake_case -> camelCase adapter so Phase 0 code stays frozen.
Spec: docs/internal/pro-regional-intelligence-upgrade.md
* feat(intelligence): get-regional-snapshot RPC handler
Reads canonical persisted RegionalSnapshot for a region via the two-hop
lookup pattern established by the Phase 0 persist layer:
1. GET intelligence:snapshot:v1:{region}:latest -> snapshot_id
2. GET intelligence:snapshot-by-id:v1:{snapshot_id} -> full snapshot JSON
Returns empty response (snapshot omitted) when:
- No latest pointer exists (seed has never run or unknown region)
- Latest pointer references a pruned or TTL-expired snapshot
- Snapshot JSON is malformed
The handler does NOT recompute on miss. One writer (the seed bundle),
canonical reads. Matches the architecture commitment in the spec.
Includes a full snake_case -> camelCase adapter so the persisted Phase 0
shape (shared/regions.types.d.ts) maps cleanly onto the camelCase proto
wire format generated by buf. The adapter is the single bridge between
the two shapes; Phase 0 code stays frozen. Adapter handles every nested
message: SnapshotMeta, RegimeState, BalanceVector (+pressures/buffers
drivers), ActorState, LeverageEdge, ScenarioSet (+lanes +transmissions),
TransmissionPath, TriggerLadder (+triggers +thresholds), MobilityState
(+airspace +flight corridors +airports), EvidenceItem, RegionalNarrative
(+5 sections +watch items).
Wiring:
- Registered on intelligenceHandler in handler.ts
- Added to PREMIUM_RPC_PATHS (src/shared/premium-paths.ts) so the
gateway enforces Pro subscription or API key
- Added to RPC_CACHE_TIER with 'slow' tier (300s browser, 1800s edge)
matching similar premium intelligence RPCs
Not in this PR:
- LLM narrative generator (follow-up PR2, wires into snapshot writer)
- RegionalIntelligenceBoard panel UI (follow-up PR3)
- ENDPOINT_ENTITLEMENTS tier-specific enforcement (PREMIUM_RPC_PATHS
alone is the Pro gate; only stock-analysis endpoints currently use
tier-specific enforcement)
* test(intelligence): unit tests for get-regional-snapshot adapter + structural checks
29 tests across 5 suites covering:
adaptSnapshot (18 tests): real unit tests of the snake_case -> camelCase
adapter with synthetic persisted snapshots. Covers every nested message
(SnapshotMeta, RegimeState, BalanceVector with 7 axes + decomposed drivers,
ActorState, LeverageEdge, ScenarioSet with nested lanes and transmissions,
TriggerLadder with all 3 buckets + TriggerThreshold, MobilityState with
airspace/flights/airports, EvidenceItem, RegionalNarrative with all 5
sections + watch_items). Also asserts empty-default behavior when
nested fields are missing.
Handler structural checks (8 tests): validates import of getCachedJson,
canonical key prefixes, two-hop lookup ordering, empty-response fallbacks
on missing pointer or malformed snapshot, and export signature matching
the service interface.
Registration (2 tests): confirms getRegionalSnapshot is imported and
registered on the intelligenceHandler object.
Security wiring (2 tests): confirms the endpoint is in PREMIUM_RPC_PATHS
and RPC_CACHE_TIER with 'slow' tier.
Proto definition (3 tests): confirms the RPC method declaration, region_id
validation regex, RegionalSnapshot top-level field layout, and
BalanceVector 7-axis declaration.
* fix(intelligence): address Greptile P2 review findings on #2951
Two P2 findings from Greptile on the RegionalSnapshot proto+RPC PR.
1) region_id regex permitted trailing and consecutive hyphens
Old: ^[a-z][a-z0-9-]*$ — accepted "mena-", "east-asia-", "foo--bar"
New: ^[a-z][a-z0-9]*(-[a-z0-9]+)*$ — strict kebab-case, every hyphen must be
followed by at least one alphanumeric character. Regenerated openapi JSON/YAML
via `make generate`. Test assertion updated to match.
2) RPC_CACHE_TIER entry looked like dead code for premium paths
Greptile flagged that `isPremium` short-circuits the tier lookup to
'slow-browser' before RPC_CACHE_TIER is consulted, so the entry is never read
at runtime. Kept the entry because `tests/route-cache-tier.test.mjs` enforces
a parity contract requiring every generated GET route to have an explicit
tier. Added a NOTE comment in gateway.ts explaining the policy, and updated
the security-wiring test with a rationale comment so future maintainers know
the entry is intentional documentation, not a stale wire.
|
||
|
Elie Habib
|
46c35e6073 | feat(breadth): add market breadth history chart (#2932) | ||
|
Elie Habib
|
55c9c36de2 |
feat(stocks): add insider transaction tracking to stock analysis panel (#2928)
* feat(stocks): add insider transaction tracking to stock analysis panel Shows 6-month insider buy/sell activity from Finnhub: total buys, sells, net value, and recent named-exec transactions. Gracefully skips when FINNHUB_API_KEY is unavailable. * fix: add cache tier entry for get-insider-transactions route * fix(stocks): add insider RPC to premium paths + fix empty/stale states * fix(stocks): add insider RPC to premium paths + fix empty/stale states - Add /api/market/v1/get-insider-transactions to PREMIUM_RPC_PATHS - Return unavailable:false with empty transactions when Finnhub has no data (panel shows "No insider transactions" instead of "unavailable") - Mark stale insider data on refresh failures to avoid showing outdated info - Update test to match new empty-data behavior * fix(stocks): unblock stock-analysis render and surface exercise-only insider activity - loadStockAnalysis no longer awaits loadInsiderDataForPanel before panel.renderAnalyses. The insider fetch now fires in parallel after the primary render at both the cached-snapshot and live-fetch call sites. When insider data arrives, loadInsiderDataForPanel re-renders the panel so the section fills in asynchronously without holding up the analyst report on a secondary Finnhub RPC. - Add transaction code 'M' (exercise / conversion of derivative) to the allowed set in get-insider-transactions so symbols whose only recent Form 4 activity is option/RSU exercises no longer appear as "No insider transactions in the last 6 months". Exercises do not contribute to buys/sells dollar totals because transactionPrice is the strike price, not a market transaction. - Panel table now uses a neutral (dim) color for non-buy/non-sell rows (M rows) instead of the buy/sell green/red binary. - Tests cover: exercise-only activity producing non-empty transactions with zero buys/sells, and blended P/S/M activity preserving all three rows. * fix(stocks): prevent cached insider fetch from clobbering live render - Cached-path insider enrichment only runs when no live fetch is coming - Added generation counter to guard against concurrent loadStockAnalysis calls - Stale insider fetches now no-op instead of reverting panel state * fix(stocks): hide transient insider-unavailable flash and zero out strike-derived values - renderInsiderSection returns empty string when insider data is not yet fetched, so the transient "Insider data unavailable" card no longer flashes on initial render before the RPC completes - Exercise rows (code M) now carry value: 0 on the server and render a dash placeholder in the Value cell, matching how the buy/sell totals already exclude strike-derived dollar amounts * fix(stocks): exclude non-market Form 4 codes (A/D/F) from insider buy/sell totals Form 4 codes A (grant/award), D (disposition to issuer), and F (tax/exercise payment) are not open-market trades and should not drive insider conviction totals. Only P (open-market purchase) and S (open-market sale) now feed the buy/sell aggregates. A/D/F rows are still surfaced in the transaction list alongside M (exercise) with value zeroed out so the panel does not look empty. |
||
|
Elie Habib
|
2b189b77b6 |
feat(stocks): add dividend growth analysis to stock analysis panel (#2927)
* feat(stocks): add dividend growth analysis to stock analysis panel Shows yield, 5Y CAGR, frequency (quarterly/monthly/annual), payout ratio, and ex-dividend date. Hidden for non-dividend stocks. Data from Yahoo Finance dividend history. * fix(stocks): bump cache key + fix partial-year CAGR + remove misleading avg yield * fix(stocks): hydrate payout ratio, drop dead five-year yield, currency-aware dividend rate - Add fetchPayoutRatio helper that calls Yahoo quoteSummary's summaryDetail module in parallel with the dividend chart fetch and returns the raw 0-1 payout ratio (or undefined when missing/non-positive). The chart endpoint alone never returns payoutRatio, which is why it was hardcoded to 0. - Make payout_ratio optional in proto and DividendProfile so a missing value is undefined instead of 0; remove five_year_avg_dividend_yield entirely (proto reserved 51) since it was always returned as 0 and never wired up. - StockAnalysisPanel.renderDividendProfile now omits the Payout Ratio cell unless the value is present and > 0, formats it as (raw * 100).toFixed(1)%, and renders the dividend rate via Intl.NumberFormat with item.currency so EUR/GBP/JPY tickers no longer get a hardcoded "$" prefix. - Bump live cache key v2 -> v3 so any cached snapshots persisted with the old shape are refetched once. - Tests cover: payoutRatio populated from summaryDetail, payoutRatio undefined when summaryDetail returns 500 or raw=0, dividend response shape no longer contains fiveYearAvgDividendYield. * fix(stocks): bump persisted history store to v3 to rotate pre-PR snapshots Live analyze-stock cache was already bumped to v3, but the persisted history store (premium-stock-store.ts) still used v2 keys for index/item lookups. Pre-PR snapshots without the new dividend fields could pass the age-only freshness check and suppress a live refetch, leaving the new dividend section missing for up to 15 minutes. Bumping the persisted store keys to v3 makes old snapshots invisible. The data loader sees an empty history, triggers a live refetch, and repopulates under the new v3 keys. Old v2 keys expire via TTL. * fix(stocks): compute dividend CAGR correctly for quarterly/semiannual/annual payers Previously computeDividendCagr() required at least 10 distinct dividend months for a year to count as "full", which excluded every non-monthly dividend payer (quarterly = 4 months, semiannual = 2, annual = 1). CAGR therefore collapsed to 0/N/A for most ordinary dividend stocks. The new check uses calendar position: any year strictly earlier than the current calendar year is treated as complete, and the current in-progress year is excluded to avoid penalizing stocks whose next payment has not yet landed. * test(stocks): pass analystData to buildAnalysisResponse after rebase onto #2926 |
||
|
Elie Habib
|
889fa62849 |
feat(stocks): add analyst consensus + price targets to stock analysis panel (#2926)
* feat(stocks): add analyst consensus + price targets to stock analysis panel Shows recommendation trend (strongBuy/buy/hold/sell), price target range (high/low/median vs current), and recent upgrade/downgrade actions with firm names. Data from Yahoo Finance quoteSummary. * chore: regenerate proto types and OpenAPI docs * fix(stocks): fallback median to mean + use stock currency for price targets * fix(stocks): drop fake $0 price targets and force refetch for pre-rollout snapshots - Make PriceTarget high/low/mean/median/current optional in proto so partial Yahoo financialData payloads stop materializing as $0.00 cells in the panel. - fetchYahooAnalystData now passes undefined (via optionalPositive) when a field is missing or non-positive, instead of coercing to 0. - StockAnalysisPanel.renderPriceTarget skips Low/High cells entirely when the upstream value is missing and falls back to a Median + Analysts view. - Add field-presence freshness check in stock-analysis-history: snapshots written before the analyst-revisions rollout (no analystConsensus and no priceTarget) are now classified as stale even when their generatedAt is inside the freshness window, so the data loader forces a live refetch. - Tests cover undefined targets path, missing financialData path, and the three field-presence freshness branches. * fix(stocks): preserve fresh snapshots on partial refetch + accept median-only targets - loadStockAnalysis now merges still-fresh cached symbols with refetched live results so a partial refetch does not shrink the rendered watchlist - renderAnalystConsensus accepts median-only price targets (not just mean) |
||
|
Elie Habib
|
a742537ae5 |
feat(supply-chain): Sprint D — GetSectorDependency RPC + vendor route-intelligence API + webhooks (#2905)
* feat(supply-chain): Sprint D — GetSectorDependency RPC + vendor route-intelligence API + webhooks * fix(supply-chain): move bypass-corridors + chokepoint-registry to server/_shared to fix api/ boundary violations * fix(supply-chain): webhooks — persist secret, fix sub-resource routing, add ownership check * fix(supply-chain): address PR #2905 review findings - Use SHA-256(apiKey) for ownerTag instead of last-12-chars (unambiguous ownership) - Implement GET /api/v2/shipping/webhooks list route via per-owner Redis Set index - Tighten SSRF: https-only, expanded metadata hostname blocklist, document DNS rebinding edge-runtime limitation - Fix get-sector-dependency.ts stale src/config/ imports → server/_shared/ (Greptile P1) * fix(supply-chain): getSectorDependency returns blank primaryChokepointId for landlocked countries computeExposures() previously mapped over all of CHOKEPOINT_REGISTRY even when nearestRouteIds was empty, producing a full array of score-0 entries in registry insertion order. The caller's exposures[0] then picked the first registry entry (Suez) as the "primary" chokepoint despite primaryChokepointExposure = 0. LI, AD, SM, BT and other landlocked countries were all silently assigned a fake chokepoint. Fix: guard at the top of computeExposures() -- return [] when input is empty so primaryChokepointId stays '' and primaryChokepointExposure stays 0. |
||
|
Elie Habib
|
ce30a48664 |
feat(resilience): add rankStable flag to ranking items (#2879)
* feat(resilience): add rankStable flag to ranking items Countries with score interval width <= 8 (p95-p05) are flagged as rankStable=true, indicating robust ranking under weight perturbation. Read from batch-computed intervals in Redis. * fix(resilience): guard inverted intervals + scope fetch to scored countries 1. isRankStable rejects negative width (malformed p05 > p95) 2. fetchIntervals scoped to cachedScores.keys() instead of all countries * fix(resilience): raw key read for intervals + bump ranking cache to v8 * fix(resilience): remove duplicate ScoreInterval interface after rebase ScoreInterval is now generated in service_server.ts (from PR #2877). Remove the local duplicate and re-export the generated type. |
||
|
Elie Habib
|
1af73975b9 |
feat(energy): SPR policy classification layer (#2881)
* feat(energy): add SPR policy classification layer with 66-country registry Static JSON registry classifying strategic petroleum reserve regimes for 66 countries (all IEA members + major producers/consumers). Integrates into energy profile handler, shock model limitations, analyst context, spine seeder, and CDP UI. - scripts/data/spr-policies.json: 66-entry registry with regime, source, asOf - scripts/seed-spr-policies.mjs: seeder following chokepoint-baselines pattern - Proto fields 51-59 on GetCountryEnergyProfileResponse - Handler reads SPR registry from Redis, populates proto fields - Shock model adds fuel-mode-gated SPR limitations for non-IEA gov SPR - Analyst context refactored to accumulator pattern (IEA + SPR parts) - CDP UI: SPR badge for non-IEA government_spr, muted text for spare_capacity - Spine integration: SPR fields in shockInputs + hasSprPolicy coverage flag - Cache keys, health, bootstrap, seed-health registrations - Tests: registry shape, ISO2, regime enum, required entries, no estimatedFillPct * fix(energy): remove SPR from bootstrap (server-only); narrow SPR hasAny gate to renderable regimes * feat(energy): render "no known SPR" risk note for countries with regime=none * fix(energy): human-readable SPR regime labels; parallelize spine+registry reads in analyst |
||
|
Elie Habib
|
0a1b74a9b2 |
feat(resilience): add score confidence intervals via batch Monte Carlo (#2877)
* feat(resilience): add score confidence intervals via batch Monte Carlo
Weekly cron perturbs domain weights ±10% across 100 draws per country,
stores p05/p95 in Redis. Score handler reads intervals and includes
them in the API response as ScoreInterval { p05, p95 }.
Proto field 14 (score_interval) added to GetResilienceScoreResponse.
* chore: regenerate proto types and OpenAPI docs for ScoreInterval
* fix(resilience): add seed-meta + lock + fix interval cache + percentile formula
1. Write seed-meta:resilience:intervals for health monitoring
2. Add distributed lock to prevent concurrent cron overlap
3. Move scoreInterval read outside 6h score cache boundary
4. Fix percentile index from floor to ceil-1 (nearest-rank)
* fix(health): add resilience:intervals to health + seed-health registries
* fix(seed): skip seed-meta on no-op runs + register intervals in health check
|
||
|
Elie Habib
|
23ed4eba44 |
fix(supply-chain): address all code review findings from PR #2873 (#2878)
* fix(supply-chain): address all code review findings from PR #2873 - Rename costIncreasePct → supplyDeficitPct (semantic correction) - Add primaryChokepointWarRiskTier to GetBypassOptionsResponse - Consolidate ThreatLevel/threatLevelToWarRiskTier into _insurance-tier.ts - Replace inline CpEntry/ChokepointStatusCacheEntry with ChokepointInfo - Add outer cachedFetchJson wrapper (3 serial Redis reads → 1 on warm path) - Add hs2 validation guard matching sibling handler pattern - Extract CHOKEPOINT_STATUS_KEY constant; eliminate string literal duplication - Add SCORE_RISK_WEIGHT/SCORE_COST_WEIGHT named constants; clamp liveScore ≥ 0 - Add Math.max(0,...) to liveScore for sub-1.0 cost multiplier corridors - Fix closurePct: req.closurePct ?? 100 (was || which falsy-coalesced zero) - Type fetchBypassOptions cargoType as CargoType (was implicit string) - Add exhaustiveness check to threatLevelToInsurancePremiumBps switch - Move TIER_RANK to module level in _insurance-tier.ts - Update WIDGET_PRO_SYSTEM_PROMPT with both new PRO RPCs * fix(supply-chain): fix supplyDeficitPct averaging and coverageDays sentinel - Remove .filter(d > 0) from productDeficits: zero-deficit products have demand and must stay in the denominator to avoid overstating the average - Clamp coverageDays = Math.max(0, effectiveCoverDays): prevents -1 net-exporter sentinel from leaking into the public API response - Update proto comment: document 0 for net exporters - Add test assertions for both contracts * chore(api-docs): regenerate OpenAPI docs for coverage_days comment update * refactor(supply-chain): use CHOKEPOINT_STATUS_KEY in chokepoint-status writer The key was extracted to cache-keys.ts in the previous commit but the primary writer (getChokepointStatus) and BOOTSTRAP_CACHE_KEYS still embedded the raw string literal. Import the constant at both sites to complete the refactor. * test: update supply-chain-v2 assertions for CHOKEPOINT_STATUS_KEY refactor Handler now imports CHOKEPOINT_STATUS_KEY as REDIS_CACHE_KEY from cache-keys.ts rather than defining a local constant. BOOTSTRAP_CACHE_KEYS also references the constant. Update source-string assertions to match the new patterns. * fix: keep BOOTSTRAP_CACHE_KEYS.chokepoints as string literal bootstrap.test.mjs enforces string-literal values in BOOTSTRAP_CACHE_KEYS via regex. CHOKEPOINT_STATUS_KEY is used in handler imports and is the primary dedup win; the static registry entry stays as-is per test contract. |
||
|
Elie Habib
|
bd07829518 |
feat(supply-chain): Sprint 2 — bypass corridor intelligence + cost shock engine (#2873)
* feat(supply-chain): Sprint 2 — bypass corridor intelligence + cost shock engine - src/config/bypass-corridors.ts: ~40 bypass corridors for all 13 chokepoints - server/supply-chain/v1/get-bypass-options.ts: PRO-gated RPC, live bypass scoring from chokepoint status cache - server/supply-chain/v1/get-country-cost-shock.ts: PRO-gated RPC, war risk premium BPS + energy coverage days (HS 27) - server/supply-chain/v1/_insurance-tier.ts: pure function, Lloyd's JWC threat → premium BPS - gateway.ts + premium-paths.ts: registered both RPCs as slow-browser + PRO-gated - src/services/supply-chain/index.ts: fetchBypassOptions + fetchCountryCostShock client methods - proto: GetBypassOptions + GetCountryCostShock messages + service registrations - tests/supply-chain-sprint2.test.mjs: 61 tests covering all new components Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com> * fix(cost-shock): call computeEnergyShockScenario directly instead of reading wrong cache key The old code read from `energy:shock:${iso2}:${chokepointId}:v1` which never matches the actual v2 cache key written by compute-energy-shock.ts. Fix by calling computeEnergyShockScenario() directly (it handles v2 caching internally) and mapping effectiveCoverDays + crude product deficitPct to the response fields. * fix(cost-shock): average refined product deficitPct instead of looking for non-existent 'crude' product --------- Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com> |
||
|
Elie Habib
|
6e401ad02f |
feat(supply-chain): Global Shipping Intelligence — Sprint 0 + Sprint 1 (#2870)
* feat(supply-chain): Sprint 0 — chokepoint registry, HS2 sectors, war_risk_tier
- src/config/chokepoint-registry.ts: single source of truth for all 13
canonical chokepoints with displayName, relayName, portwatchName,
corridorRiskName, baselineId, shockModelSupported, routeIds, lat/lon
- src/config/hs2-sectors.ts: static dictionary for all 99 HS2 chapters
with category, shockModelSupported (true only for HS27), cargoType
- server/worldmonitor/supply-chain/v1/_chokepoint-ids.ts: migrated to
derive CANONICAL_CHOKEPOINTS from chokepoint-registry; no data duplication
- src/config/geo.ts + src/types/index.ts: added chokepointId field to
StrategicWaterway interface and all 13 STRATEGIC_WATERWAYS entries
- src/components/MapPopup.ts: switched chokepoint matching from fragile
name.toLowerCase() to direct chokepointId === id comparison
- server/worldmonitor/intelligence/v1/_shock-compute.ts: migrated from old
IDs (hormuz/malacca/babelm) to canonical IDs (hormuz_strait/malacca_strait/
bab_el_mandeb); same for CHOKEPOINT_LNG_EXPOSURE
- proto/worldmonitor/supply_chain/v1/supply_chain_data.proto: added
WarRiskTier enum + war_risk_tier field (field 16) on ChokepointInfo
- get-chokepoint-status.ts: populates warRiskTier from ChokepointConfig.threatLevel
via new threatLevelToWarRiskTier() helper (FREE field, no PRO gate)
* feat(supply-chain): Sprint 1 — country chokepoint exposure index + sector ring
S1.1: scripts/shared/country-port-clusters.json
~130 country → {nearestRouteIds, coastSide} mappings derived from trade route
waypoints; covers all 6 seeded Comtrade reporters plus major trading nations.
S1.2: scripts/seed-hs2-chokepoint-exposure.mjs
Daily cron seeder. Pure computation — reads country-port-clusters.json,
scores each country against CHOKEPOINT_REGISTRY route overlap, writes
supply-chain:exposure:{iso2}:{hs2}:v1 keys + seed-meta (24h TTL).
S1.3: RPC get-country-chokepoint-index (PRO-gated, request-varying)
- proto: GetCountryChokepointIndexRequest/Response + ChokepointExposureEntry
- handler: isCallerPremium gate; cachedFetchJson 24h; on-demand for any iso2
- cache-keys.ts: CHOKEPOINT_EXPOSURE_KEY(iso2, hs2) constant
- health.js: chokepointExposure SEED_META entry (48h threshold)
- gateway.ts: slow-browser cache tier
- service client: fetchCountryChokepointIndex() exported
S1.4: Chokepoint popup HS2 sector ring chart (PRO-gated)
Static trade-sector breakdown (IEA/UNCTAD estimates) per 9 major chokepoints.
SVG donut ring + legend shown for PRO users; blurred lockout + gate-hit
analytics for free users. Wired into renderWaterwayPopup().
🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.com/claude-code) + Compound Engineering v2.49.0
Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* fix(tests): update energy-shock-v2 tests to use canonical chokepoint IDs
CHOKEPOINT_EXPOSURE and CHOKEPOINT_LNG_EXPOSURE keys were migrated from
short IDs (hormuz, malacca, babelm) to canonical registry IDs
(hormuz_strait, malacca_strait, bab_el_mandeb) in Sprint 0.
Test fixtures were not updated at the time; fix them now.
* fix(tests): update energy-shock-seed chokepoint ID to canonical form
VALID_CHOKEPOINTS changed to canonical IDs in Sprint 0; the seed test
that checks valid IDs was not updated alongside it.
* fix(cache-keys): reword JSDoc comment to avoid confusing bootstrap test regex
The comment "NOT in BOOTSTRAP_CACHE_KEYS" caused the bootstrap.test.mjs
regex to match the comment rather than the actual export declaration,
resulting in 0 entries found. Rephrase to "excluded from bootstrap".
* fix(supply-chain): address P1 review findings for chokepoint exposure index
- Add get-country-chokepoint-index to PREMIUM_RPC_PATHS (CDN bypass)
- Validate iso2/hs2 params before Redis key construction (cache injection)
- Fix seeder TTL to 172800s (2× interval) and extend TTL on skipped lock
- Fix CHOKEPOINT_EXPOSURE_SEED_META_KEY to match seeder write key
- Render placeholder sectors behind blur gate (DOM data leakage)
- Document get-country-chokepoint-index in widget agent system prompts
* fix(lint): resolve Biome CI failures
- Add biome.json overrides to silence noVar in HTML inline scripts,
disable linting for public/ vendor/build artifacts and pro-test/
- Remove duplicate NG and MW keys from country-port-clusters.json
- Use import attributes (with) instead of deprecated assert syntax
* fix(build): drop JSON import attribute — esbuild rejects `with` syntax
---------
Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
|
||
|
Elie Habib
|
cffdcf8052 |
fix(energy): V6 review findings (7 issues across 5 PRs) (#2861)
* fix(energy): V6 review findings — flow availability, ember proto bool, LNG stale/blocking, fixture accuracy
- Fix 1: Only show "Flow data unavailable" for chokepoints with PortWatch
flow coverage (hormuz, malacca, suez, bab_el_mandeb), not all corridors
- Fix 2: Correct proto comment on data_available field 9 to document
gas mode and both mode behavior
- Fix 3: Add ember_available bool field 50 to GetCountryEnergyProfile proto,
set server-side from spine.electricity or direct Ember key fallback
- Fix 4: Ember fallback reads energy:ember:v1:{code} when spine exists but
has no electricity block (or fossilShare is absent)
- Fix 6: Add IEA upstream fixture matching actual API response shape,
with golden test parsing through seeder parseRecord/buildIndex
- Fix 7: Add PortWatch ArcGIS fixture with all attributes.* fields used
by buildHistory, with golden test validating parsed output
* fix(energy): add emberAvailable to energy gate; use real buildHistory in portwatch test
* fix(energy): add Ember render block to renderEnergyProfile for Ember-only countries
* chore: regenerate OpenAPI specs after proto comment update
|
||
|
Elie Habib
|
75e9c22dd3 |
feat(resilience): populate dataVersion field from seed-meta timestamp (#2865)
* feat(resilience): populate dataVersion field from seed-meta timestamp Sets dataVersion to the ISO date of the most recent static bundle seed, making the data vintage visible to API consumers. * fix(resilience): bump score cache to v7 for dataVersion field addition |
||
|
Elie Habib
|
c35b3618e0 |
feat(fires): flag possible explosions in satellite thermal detections (#2850)
* feat(fires): flag possible explosions in satellite thermal detections Adds possibleExplosion field (FRP >80 MW + brightness >380 K) to fire detections, surfacing non-fire thermal signatures that may indicate strikes or explosions. Seeder computes the flag, panel shows inline badge per region and summary alert when explosions are detected. * refactor(fires): extract brightness/frp locals to avoid double-parse |
||
|
Elie Habib
|
c826003e5f |
feat(energy): LNG and gas shock path (V5-4) (#2822)
* feat(energy): LNG gas shock path with GIE storage buffer (V5-4) * fix(energy): gas path zero-LNG handling, oil gate bypass, live flow scaling, coverage semantics * fix(energy): gas-only coverage respects degraded; zero oil fields in gas mode |
||
|
Elie Habib
|
f53c05599a |
feat(resilience): baseline vs stress scoring engine (#2821)
* feat(resilience): baseline vs stress scoring engine Splits the resilience index into structural capacity (baselineScore) and active disruption (stressScore) using the dimension type tags from RESILIENCE_DIMENSION_TYPES (baseline/stress/mixed). overallScore = baselineScore * (1 - stressFactor) where stressFactor is clamped to [0, 0.5]. Mixed dimensions contribute to both scores. Proto fields 10-12 added (baseline_score, stress_score, stress_factor). Widget updated to display baseline/stress breakdown. Cache keys bumped v4 -> v5 for atomic rollout. * fix(resilience): bump history key to v2 for baseline/stress formula change The overallScore formula changed from domain-weighted-sum to baselineScore * (1 - stressFactor). Old history entries are incomparable, causing fake change30d drops of -20 to -30 points. Versioned history key starts a clean series. |
||
|
Elie Habib
|
80b24d8686 |
feat(energy): shock model v2 — live flow ratios, coverage, limitations (V5-3) (#2810)
* feat(energy): shock model v2 — live flow ratios, coverage, limitations (V5-3) Replace static CHOKEPOINT_EXPOSURE multipliers with live PortWatch flow ratios from energy:chokepoint-flows:v1. Add proto fields 11-19: per-source coverage flags (jodi_oil_coverage, comtrade_coverage, iea_stocks_coverage, portwatch_coverage), coverage_level, limitations[], degraded, chokepoint_confidence, live_flow_ratio. Expand ISO2_TO_COMTRADE from 6 to 150+ countries via _comtrade-reporters.ts. Partial-coverage path proxies Gulf share at 40%. Unsupported countries return structured metadata instead of opaque string. data_available (field 9) preserved for backward compat. * fix(energy): correct chokepoint-flows key shape in shock handler (V5-3) energy:chokepoint-flows:v1 is a flat object keyed by canonicalId (hormuz_strait, bab_el_mandeb, suez, malacca_strait), not an object with a chokepoints[] array. The wrong shape caused degraded=true and liveFlowRatio=null for every request, silently falling back to static CHOKEPOINT_EXPOSURE multipliers even when live PortWatch data was available. Fixes: ChokepointEntry interface, typecast, cpEntry lookup, degraded check. * fix(energy): 4 review fixes for shock v2 handler (V5-3) 1. Cache key v1→v2: response shape changed (fields 11-19 added), old v1 cache entries would be served without new coverage fields. 2. IEA unknown vs zero: when ieaStocksCoverage=false, assessment now shows "IEA cover: unknown" instead of "0 days" to avoid conflating missing data with real zero stock. 3. liveFlowRatio 0.0 truthiness: changed `if (liveFlowRatio)` to `if (liveFlowRatio != null)` — a blocked chokepoint (ratio=0.0) is now shown, not hidden as "no live data". 4. Badge cleared on request start: coverageBadge is now reset before each shock compute request so a failed request doesn't leave the previous result's badge visible. * fix(energy): 3 remaining review fixes for shock v2 (V5-3) 1. Flow column: gate on portwatchCoverage (bool) not liveFlowRatio!=null — proto double defaults to 0, so null-check was always true and degraded responses showed a misleading "Flow: 0%" column. 2. Degraded cache TTL: 5min when degraded=true, 1h when live data present — limits how long stale degraded state persists after PortWatch recovers. 3. IEA anomaly cover days: zero daysOfCover when ieaStocksCoverage=false so anomalous IEA data no longer contributes to effectiveCoverDays; panel IEA cover row also gated on ieaStocksCoverage. * fix(energy): netExporter from anomalous IEA + zero-days panel display (V5-3) 1. netExporter: gated on ieaStocksCoverage — anomalous IEA rows no longer drive the net-exporter assessment branch when coverage=false. 2. Panel IEA cover: removed effectiveCoverDays>0 guard so a real zero (reserves exhausted under scenario) renders as "0 days" instead of being silently hidden as if there were no IEA data. * fix(energy): handle net-exporter sentinel (-1) in IEA cover panel row * fix(energy): NaN guard on flowRatio; optional live_flow_ratio; coverageLevel includes IEA+degraded * fix(energy): narrow Gulf-share proxy to Comtrade-only; NaN guard computeGulfShare; EMPTY liveFlowRatio undefined * fix(energy): tighten ieaStocksCoverage null guard; cache key varies by degraded state * fix(energy): harden IEA/PortWatch input validation; reduce shock cache TTL * fix(energy): add null narrowing for daysOfCover to satisfy strict TS |
||
|
Elie Habib
|
1937dbf844 |
feat(portwatch): Maritime Activity section in CountryDeepDivePanel (PR C) (#2805)
* feat(portwatch): Maritime Activity section in CountryDeepDivePanel (PR C) - Add get_country_port_activity.proto with PortActivityEntry + CountryPortActivityResponse messages - Register GetCountryPortActivity RPC in service.proto with HTTP GET /get-country-port-activity path - Run make generate to produce updated service_client.ts and service_server.ts - Implement get-country-port-activity.ts handler: countries guard, top-5 slice, trendDelta→tankerCallsPrev mapping - Register handler in intelligence handler.ts and gateway.ts slow cache tier - Add CountryPortActivityData interface and updateMaritimeActivity? method to CountryBriefPanel - Implement updateMaritimeActivity in CountryDeepDivePanel: table with 5 ports, anomaly badge, trend color, IMF PortWatch footer - Add getCountryPortActivity call in country-intel.ts with stale guard - Add maritime-activity CMD+K entry in commands.ts - 29 source-string assertions in tests/country-port-activity.test.mjs (all pass) Task: PR C * fix(portwatch): pass trendDelta directly, add runtime trend tests |
||
|
Elie Habib
|
2edcdeee06 |
refactor(resilience): remove Cronbach alpha, add imputationShare confidence (#2787)
* refactor(resilience): remove Cronbach alpha, add imputationShare confidence Remove cronbach_alpha from proto (field 5 reserved) and all response builders. Replace with imputationShare (field 9): the fraction of weighted score from imputed (not observed) data. lowConfidence now triggers on averageCoverage < 0.55 or imputationShare > 0.40, replacing the unstable Cronbach-based gate. weightedBlend() return type extended with observedWeight/imputedWeight for provenance tracking through the scoring pipeline. * fix(resilience): version cache key + fix IMF proxy imputation classification 1. Bump resilience score cache key to v2 to avoid serving stale cached responses missing imputationShare after deploy. 2. Add explicit `imputed` flag to WeightedMetric so proxy data (real IMF inflation with lower certaintyCoverage) is classified as observed, not imputed. Only synthetic absence-based scores count toward imputationShare. |
||
|
Elie Habib
|
a09f49ff9c |
feat(supply-chain): energy flow estimates per chokepoint (mb/d card row) (#2780)
* feat(supply-chain): energy flow estimates per chokepoint (mb/d card row) - Add FlowEstimate proto message + ChokepointInfo field 15; regenerate stubs - Add baselineId mapping to _chokepoint-ids.ts (7 of 13 chokepoints) - Add relayId to seed-chokepoint-baselines.mjs CHOKEPOINTS entries - New seed-chokepoint-flows.mjs: reads portwatch + baselines, computes 7d tanker avg vs 90d baseline, outputs flow_ratio and current_mbd; prefers DWT (capTanker) when available; flags disruption if last 3 days each below 0.85 threshold; writes energy:chokepoint-flows:v1 (TTL 3d) - get-chokepoint-status.ts: parallel-reads flows key, attaches flowEstimate - SupplyChainPanel: compact card gains mb/d row (red <85%, amber <95%) - 19 new unit tests for flow computation and seeder contract * fix(chokepoint-flows): base useDwt on 90d baseline window, not recent 7 days Zero recent capTanker is the disruption signal, not a reason to fall back to vessel counts. Switching metrics during peak disruption caused the seeder to report a higher (less accurate) flow estimate exactly when oil-flow collapse is most acute. useDwt is now locked to whether the baseline window has DWT data -- stable across disruption events. Adds regression test covering DWT-collapse scenario. * fix(chokepoint-flows): require majority DWT coverage in baseline before activating DWT mode capBaselineSum > 0 would activate DWT on a single non-zero day during partial data roll-out, pulling down the baseline average via zero-filled gaps. Now requires >= ceil(prev90.length / 2) days with DWT data. ArcGIS data is all-or-nothing per chokepoint in practice, so this guard catches edge cases without affecting normal operation. |
||
|
Elie Habib
|
190095ca89 |
feat(supply-chain): stacked vessel-type transit chart with 7d MA, DWT tab, zoom (#2777)
* feat(supply-chain): stacked vessel-type transit chart with 7d MA, DWT tab, zoom - Update TransitDayCount proto (fields 6-14): container, dry_bulk, general_cargo, roro, cap_* DWT capacity fields; regenerate TS types - Rewrite transit-chart.ts: 5-type stacked bar (container/dryBulk/ generalCargo/roro/tanker), 7d MA dashed overlay, Transit Calls / Trade Volume tab toggle, 1m/3m/6m zoom buttons, richer tooltip - SupplyChainPanel: enlarge chart placeholder min-height 120->200px * fix(transit-chart): stop control clicks bubbling + track source div in destroy - stopPropagation on controls container prevents tab/zoom button clicks from collapsing the chokepoint card - source div now tracked as this.source and cleaned up in destroy(), preventing duplicate attribution lines on repeated remounts * fix(transit-chart): import from generated client, reuse data in onMouseMove - Import TransitDayCount from generated client stub instead of server layer; keeps src/ imports within src/ - onMouseMove: reuse already-bound data array for MA computation instead of calling visibleData() again on every mouse event |
||
|
Elie Habib
|
b9b552cfcd |
feat(energy): product supply shock scenario RPC (Phase 4 PR C) (#2768)
* feat(energy): ComputeEnergyShockScenario RPC + country brief shock UI (Phase 4 PR C) Adds on-demand product supply shock scenario computation from JODI Oil, Comtrade and IEA stocks data. * fix(tests): add runtime + intelligence-client stubs to resilience harness ResilienceWidget imports @/services/runtime which dynamically imports @/services/widget-store. Without stubbing runtime, esbuild bundled the chain and failed on loadFromStorage not exported by the utils stub. * fix(energy): dataAvailable requires Comtrade data; gate shock widget on jodiOilAvailable - dataAvailable now requires both jodiOil and comtradeHasData, eliminating the contradiction of returning true with an "insufficient data" assessment - Collapsed redundant !hasComtradeData branch into the unified !dataAvailable guard - Gate renderShockScenarioWidget() behind data.jodiOilAvailable in CountryDeepDivePanel to avoid rendering a widget that will always return dataAvailable: false * fix(energy): zero-import hasData=false; extract pure shock-compute for real unit coverage - `totalImports === 0` in `computeGulfShare` now returns `hasData: false` so the handler correctly falls through to the "insufficient data" branch instead of treating empty Comtrade rows as usable Gulf-share data - Extract `clamp`, `computeGulfShare`, `computeEffectiveCoverDays`, `buildAssessment`, `GULF_PARTNER_CODES`, and `CHOKEPOINT_EXPOSURE` into `_shock-compute.ts` - Handler delegates pure computation to imported functions; `getGulfCrudeShare` still owns Redis I/O and calls `computeGulfShare(flows)` for the math - Tests now import the real functions via `.js` ESM extension; all 24 test cases exercise actual production logic (was previously reimplemented inline) * fix(energy): handle net-exporter in shock assessment; fix tautological chokepoint tests * fix(energy): move net-exporter branch before low-Gulf-share check in buildAssessment A net exporter with gulfCrudeShare < 0.1 (e.g. Norway) incorrectly received "low Gulf crude dependence" instead of "net oil exporter". Adds regression test to cover the ordering case. |
||
|
Elie Habib
|
e0dc630ed5 |
feat(energy): days of cover global view (Phase 4 PR B) (#2767)
* feat(energy): days of cover analysis key + EnergyComplexPanel oil stocks section - seed-iea-oil-stocks.mjs exports buildOilStocksAnalysis and writes energy:oil-stocks-analysis:v1 via afterPublish hook after main index - Rankings sorted by daysOfCover desc (net-exporters last), vsObligation, obligationMet, regional summaries (Europe/Asia-Pacific/North America) - EnergyComplexPanel.setOilStocksAnalysis() renders IEA member table with below-obligation badges, rank, days vs 90d obligation, regional summary rows - Health monitoring: seed-meta:energy:oil-stocks-analysis (42d maxStaleMin) - Gateway cache tier: static (monthly seed data) - 13 new tests covering sorting, exclusions, regional rollups, obligation logic * feat(energy): add proto + regenerate service for oil stocks analysis RPC - Add get_oil_stocks_analysis.proto with OilStocksAnalysisMember, OilStocksRegionalSummary sub-messages, and GetOilStocksAnalysisResponse - Use proto3 optional fields for nullable int32 (daysOfCover, vsObligation, avgDays, minDays) avoiding google.protobuf.wrappers complexity - Regenerate service_client.ts + service_server.ts via make generate - Update handler fallback and panel null-safety guards for optional fields - Regenerated OpenAPI docs include getOilStocksAnalysis endpoint * fix(energy): preserve oil-stocks-analysis TTL via extraKeys; fix seed-meta TTL to exceed health threshold - Move ANALYSIS_KEY into ANALYSIS_EXTRA_KEY in extraKeys so runSeed() extends its TTL on fetch failure or validation skip (was only written in afterPublish, leaving the key unprotected on the sad path) - afterPublish now writes only the seed-meta for ANALYSIS_KEY with a 50-day TTL (Math.max(86400*50, TTL_SECONDS)) — exceeds the health maxStaleMin threshold - Add optional metaTtlSeconds param to writeExtraKeyWithMeta() (backward-compat, defaults to existing 7-day value for all other callers) - Update health.js oilStocksAnalysis maxStaleMin from 42d to 50d to stay below the new seed-meta TTL and avoid false stale/missing reports * fix(energy): preserve seed-meta:oil-stocks-analysis TTL via extraKeys on seeder failure |
||
|
Elie Habib
|
3e9556c37f |
feat(resilience): Phase 1 §3.2+§3.3 — full-country sanctions counts + grey-out ranking (#2763)
* feat(resilience): Phase 1 §3.2+§3.3 — full-country sanctions counts + grey-out ranking §3.2 — Switch sanctions from top-12 to full country counts - RESILIENCE_SANCTIONS_KEY: 'sanctions:pressure:v1' → 'sanctions:country-counts:v1' - New key is a plain ISO2→entryCount map covering ALL countries (no top-12 truncation) - Replaces compound pressure formula with normalizeSanctionCount() piecewise scale: 0=100, 1-10=90-75, 11-50=75-50, 51-200=50-25, 201+=25→0 - IMPUTE.ofacSanctions removed (country-counts covers all countries; no absent-country imputation needed for sanctions) §3.3 — Grey-out criteria for ranking - Proto: ResilienceRankingItem.overall_coverage (field 5) + GetResilienceRankingResponse.greyed_out - GREY_OUT_COVERAGE_THRESHOLD = 0.40: countries below this are excluded from ranking but still appear on choropleth in "insufficient data" style - buildRankingItem() now computes overallCoverage from domain/dimension data - getResilienceRanking() splits items into ranked (≥0.40) + greyedOut (<0.40) Tests updated for new sanctions format; overall score anchor updated (67.56). * fix(resilience): fix ranking cache guard for all-greyed-out + stale shape cases Two cache bugs: 1. Empty-items guard: `cached?.items?.length` fails when every country falls below GREY_OUT_COVERAGE_THRESHOLD (items=[], greyedOut=[…]). The cache was written correctly but never served, causing unnecessary rewarming on every request for sparse-data deployments. Fix: `cached != null && (items.length > 0 || greyedOut.length > 0)` 2. Stale-shape test: agent-written cache test stored a payload without `greyedOut` or `overallCoverage`, locking in pre-PR shape. Updated to the correct post-deploy shape so the test reflects actual cached content. Cache key was already bumped to resilience:ranking:v2 (forces fresh compute on first post-deploy request, avoiding old-shape responses in production). * fix(resilience): consume greyedOut on choropleth; version ranking cache key - Add 'insufficient_data' level to ResilienceChoroplethLevel and RESILIENCE_CHOROPLETH_COLORS - Extend buildResilienceChoroplethMap to accept optional greyedOut array - Thread greyedOut through DeckGLMap.setResilienceRanking, MapContainer.setResilienceRanking (with replay), and data-loader.ts - Add 'Insufficient data' tooltip guard for greyed-out countries in DeckGLMap - Bump RESILIENCE_RANKING_CACHE_KEY to resilience:ranking:v2 to invalidate stale schema-mismatched cache entries - Update api/health.js probe key to match * fix(resilience): include greyedOut in seed-meta count to avoid false health alert seed-meta:resilience:ranking was written with count=response.items.length, which excludes greyedOut countries. In an all-greyed-out deployment, count=0 causes api/health.js to report the ranking as EMPTY_DATA/critical even though the cached payload is valid (items:[], greyedOut:[…]). Fix: count = items.length + greyedOut.length — total scoured countries regardless of ranking eligibility. * test(resilience): pin all-greyed-out cache-hit regression Adds the missing test case: cached payload with items=[] and greyedOut=[…] must be served from cache without triggering score rewarming. Previously, `cached?.items?.length` was falsy for this shape, making the guard ineffective. The fix (items.length > 0 || greyedOut.length > 0) was correct but unpinned — this test locks it in. |
||
|
Elie Habib
|
5dbc72d7c6 |
feat(energy): GetCountryEnergyProfile RPC — aggregate Phase 1/2/2.5 data per country (#2747)
* feat(energy): GetCountryEnergyProfile RPC — aggregate Phase 1/2/2.5 data per country Add new intelligence RPC that reads 6 Redis keys in parallel (OWID mix, EU gas storage, electricity prices, JODI oil, JODI gas, IEA oil stocks) and returns a unified energy profile per ISO2 country code. All signals are optional with graceful omission on missing/null keys. Also register the new route in gateway.ts with slow cache tier. * fix(energy): convert gasLngShare from 0-1 fraction to 0-100 percentage to match all other share fields * fix(energy): add lpgImportsKbd, fix electricityAvailable source/date consistency, drop false US electricity claim * chore(proto): regenerate OpenAPI specs after energy profile field description update |
||
|
Fayez Bast
|
8609ad1384 |
feat: " climate disasters alerts seeders " (#2550)
* Revert "Revert "feat(climate): add climate disasters seed + ListClimateDisast…"
This reverts commit
|
||
|
Elie Habib
|
4e9f25631c |
feat(economic): add FAO Food Price Index panel (#2682)
* feat(economic): add FAO Food Price Index panel Adds a new panel tracking the FAO Global Food Price Index (FFPI) for the past 12 months, complementing existing consumer prices, fuel prices, and Big Mac Index trackers. - proto: GetFaoFoodPriceIndex RPC with 6-series response (Food, Cereals, Meat, Dairy, Oils, Sugar + MoM/YoY pct) - seeder: seed-fao-food-price-index.mjs with 90-day TTL (3× monthly), isMain guard, parseVal NaN safety, correct 13-point slice - handler/gateway: static tier RPC wired into economicHandler - bootstrap/health: bootstrapped as SLOW_KEY; maxStaleMin=86400 (60 days) - panel: SVG multi-line chart with 6 series, auto-scaled Y axis, headline with MoM/YoY indicators, info tooltip, bootstrap hydration - CMD+K: panel:fao-food-price-index with fao/ffpi/food keywords - Railway: fao-ffpi cron seeder service (0.5 vCPU, 0.5 GB, daily 08:45) - locales: full en.json keys for panel UI strings - ais-relay: faoFoodPriceIndex added to economic bootstrap context * fix(economic): add faoFoodPriceIndex to cache-keys.ts BOOTSTRAP_CACHE_KEYS and BOOTSTRAP_TIERS * fix(economic): correct cron comment in fao seeder to reflect daily schedule |
||
|
Fayez Bast
|
02f55dc584 |
feat(climate): add ocean ice indicators seed and RPC (#2652)
* feat(climate): add ocean ice indicators seed and RPC * fix(review): restore MCP maxStaleMin, widen health threshold, harden sea level parser, type globe.gl shim - Restore get_climate_data _maxStaleMin to 2880 (was accidentally lowered to 1440) - Bump oceanIce SEED_META maxStaleMin from 1440 to 2880 (2× daily interval, tolerates one missed run) - Add fallback regex patterns for NASA sea level overlay HTML parsing - Replace globe.gl GlobeInstance `any` with typed interface (index sig stays `any` for Three.js compat) * fix(review): merge prior cache on partial failures, fix fallback regex, omit trend without baseline - P1: fetchOceanIceData() now reads prior cache and merges last-known-good indicators when any upstream source fails, preventing partial overwrites from erasing previously healthy data - P1: sea level fallback regex now requires "current" context to avoid matching the historical 1993 baseline rate instead of the current rate - P2: classifyArcticTrend() returns null (omitted from payload) when no climatology baseline exists, instead of misleadingly labeling as "average" - Added tests for all three fixes * fix(review): merge prior cache by source field group, not whole object Prior-cache merge was too coarse: Object.assign(payload, priorCache) reintroduced stale arctic_extent_anomaly_mkm2 and arctic_trend from prior cache when sea-ice succeeded but intentionally omitted those fields (no climatology baseline), and an unrelated source like OHC or sea level failed in the same run. Fix: define per-source field groups (seaIce, seaLevel, ohc, sst). Only fall back to prior cache fields for groups whose source failed entirely. When a source succeeds, only its returned fields appear in the payload, even if it omits fields it previously provided. Added test covering the exact combined case: sea-ice climatology unavailable + unrelated source failure + prior-cache merge enabled. --------- Co-authored-by: Elie Habib <elie.habib@gmail.com> |
||
|
Lucas Passos
|
4b67012260 |
feat(resilience): add service proto and stub handlers (#2657)
* feat(resilience): add service proto and stub handlers Add the worldmonitor.resilience.v1 proto package, generated client/server artifacts, edge routing, and zero-state handler stubs so the domain is deployable before the seed and scoring layers land. Validation: - PATH="/Users/lucaspassos/go/bin:/Users/lucaspassos/.codex/tmp/arg0/codex-arg06nbVvG:/Users/lucaspassos/.antigravity/antigravity/bin:/Users/lucaspassos/.local/bin:/Users/lucaspassos/.codeium/windsurf/bin:/Users/lucaspassos/Library/Python/3.12/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/pkg/env/active/bin:/opt/pmk/env/global/bin:/Library/Apple/usr/bin:/opt/homebrew/bin:/Applications/Codex.app/Contents/Resources" make generate - PATH="/Users/lucaspassos/go/bin:/Users/lucaspassos/.codex/tmp/arg0/codex-arg06nbVvG:/Users/lucaspassos/.antigravity/antigravity/bin:/Users/lucaspassos/.local/bin:/Users/lucaspassos/.codeium/windsurf/bin:/Users/lucaspassos/Library/Python/3.12/bin:/Library/Frameworks/Python.framework/Versions/3.12/bin:/usr/local/bin:/System/Cryptexes/App/usr/bin:/usr/bin:/bin:/usr/sbin:/sbin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/local/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/bin:/var/run/com.apple.security.cryptexd/codex.system/bootstrap/usr/appleinternal/bin:/opt/pkg/env/active/bin:/opt/pmk/env/global/bin:/Library/Apple/usr/bin:/opt/homebrew/bin:/Applications/Codex.app/Contents/Resources" npx tsx --test tests/route-cache-tier.test.mjs tests/edge-functions.test.mjs - npm run typecheck (fails on upstream Dodo/Clerk baseline) - npm run typecheck:api (fails on upstream vitest baseline) - npm run test:data (fails on upstream dodopayments-checkout baseline via tests/runtime-config-panel-visibility.test.mjs) * fix(resilience): add countryCode validation to get-resilience-score Throw ValidationError when countryCode is missing instead of silently returning a zero-state response with an empty string country code. * fix(resilience): validate countryCode format and mark required in spec - Trim whitespace and reject non-ISO-3166-1 alpha-2 codes to prevent cache pollution from malformed aliases (e.g. 'USA', ' us ', 'foobar') - Add required: true to proto QueryConfig so generated OpenAPI spec matches runtime validation behavior - Regenerated OpenAPI artifacts via make generate --------- Co-authored-by: Elie Habib <elie.habib@gmail.com> |
||
|
Fayez Bast
|
9d94ad36aa |
feat(climate+health):add shared air quality seed and mirrored health (#2634)
* feat(climate+health):add shared air quality seed and mirrored health/climate RPCs * feat(climate+health):add shared air quality seed and mirrored health/climate RPCs * fix(air-quality): address review findings — TTL, seed-health, FAST_KEYS, shared meta - Raise CACHE_TTL from 3600 to 10800 (3× the 1h cron cadence; gold standard) - Add health:air-quality to api/seed-health.js SEED_DOMAINS so monitoring dashboard tracks freshness - Remove climateAirQuality and healthAirQuality from FAST_KEYS (large station payloads; load in slow batch) - Point climateAirQuality SEED_META to same meta key as healthAirQuality (same seeder run, one source of truth) * fix(bootstrap): move air quality keys to SLOW tier — large station payloads avoid critical-path batch * fix(air-quality): fix malformed OpenAQ URL and remove from bootstrap until panel exists - Drop deprecated first URL attempt (parameters=pm25, order_by=lastUpdated, sort=desc); use correct v3 params (parameters_id=2, sort_order=desc) directly — eliminates guaranteed 4xx retry cycle per page on 20-page crawl - Remove climateAirQuality and healthAirQuality from BOOTSTRAP_CACHE_KEYS, SLOW_KEYS, and BOOTSTRAP_TIERS — no panel consumes these yet; adding thousands of station records to every startup bootstrap is pure payload bloat - Remove normalizeAirQualityPayload helpers from bootstrap.js (no longer called) - Update service wrappers to fetch via RPC directly; re-add bootstrap hydration when a panel actually needs it * fix(air-quality): raise lock TTL to 3600s to cover 20-page crawl worst case 2 OpenAQ calls × 20 pages × (30s timeout × 3 attempts) = 3600s max runtime. Previous 600s TTL allowed concurrent cron runs on any degraded upstream. --------- Co-authored-by: Elie Habib <elie.habib@gmail.com> |
||
|
Elie Habib
|
6c017998d3 |
feat(e3): story persistence tracking (#2620)
* feat(e3): story persistence tracking
Adds cross-cycle story tracking layer to the RSS digest pipeline:
- Proto: StoryMeta message + StoryPhase enum on NewsItem (fields 9-11).
importanceScore and corroborationCount stubs added for E1.
- list-feed-digest.ts: builds corroboration map across ALL items before
truncation; batch-reads existing story:track hashes from Redis; writes
HINCRBY/HSET/HSETNX/SADD/EXPIRE per story in 80-story pipeline chunks;
attaches StoryMeta (firstSeen, mentionCount, sourceCount, phase) to
each proto item using read-back data.
- cache-keys.ts: STORY_TRACK_KEY_PREFIX, STORY_SOURCES_KEY_PREFIX,
DIGEST_ACCUMULATOR_KEY_PREFIX, STORY_TRACKING_TTL_S.
- src/types/index.ts: StoryMeta, StoryPhase, NewsItem extended.
- data-loader.ts: protoItemToNewsItem maps STORY_PHASE_* → client phase.
- NewsPanel.ts: BREAKING/DEVELOPING/ONGOING phase badges in item rows.
New story first appearance: phase=BREAKING. After 2 mentions within 2h:
DEVELOPING. After 6+ mentions or >2h: SUSTAINED. If score drops below
50% of peak: FADING (used by E1; defaults to SUSTAINED for now).
Redis keys per story (48h TTL):
story:track:v1:<hash16> → hash (firstSeen,lastSeen,mentionCount,...)
story:sources:v1:<hash16> → set (feed names, for cross-source count)
* fix(e3): correct storyMeta staleness and mentionCount semantics
P1 — storyMeta was always one cycle behind because storyTracks was read
before writeStoryTracking ran. Fix: keep read-before-write but compute
storyMeta from merged in-memory state (stale.mentionCount + 1, fresh
sourceCount from corroborationMap). New stories get mentionCount=1 and
phase=BREAKING in the same cycle they first appear — no extra Redis
round-trip needed.
P2 — mentionCount incremented once per item occurrence, so a story seen
in 3 sources in its first cycle was immediately stored as mentionCount=3.
Fix: deduplicate by titleHash in writeStoryTracking so each unique story
gets exactly one HINCRBY per digest cycle regardless of source count.
SADD still collects all sources for the set key.
* fix(e3): Unicode hash collision, ALERT badge regression, FADING comment
P1 — normalizeTitle used [^\w\s] without the u flag; \w is ASCII-only
so every Arabic/CJK/Cyrillic title stripped to "" and shared one Redis
hash. Fixed: use /[^\p{L}\p{N}\s]/gu (Unicode property escapes require
the u flag).
P1 — ALERT badge was gated on !item.storyMeta, suppressing the indicator
for any tracked story regardless of isAlert. Phase and alert are
orthogonal signals; ALERT now renders unconditionally when isAlert=true.
P2 — FADING branch is intentionally inactive until E1 ships real scores
(currentScore/peakScore placeholder 0 via HSETNX). Added comment to
document the intentional ordering.
* fix(news-alerts): skip sustained/fading stories in breaking alert selectBest
Sustained and fading story phases are already well-covered by the feed;
only breaking and developing phases warrant a banner interrupt. Items
without storyMeta (phase unspecified) pass through unchanged.
Fixes gap C from docs/plans/2026-04-02-003-fix-news-alerts-pr-gaps-plan.md
* fix(e3): remove rebase artifacts from list-feed-digest
Removes a stray closing brace, duplicate ASCII normalizeTitle
(Unicode-aware version from the fix commit is correct), and
a leftover storyPhase assignment that references a removed field.
All typecheck and typecheck:api pass clean.
|
||
|
Elie Habib
|
8d8cf56ce2 |
feat(scoring): composite importance score + story tracking infrastructure (#2604)
* feat(scoring): add composite importance score + story tracking infrastructure
- Extract SOURCE_TIERS/getSourceTier to server/_shared/source-tiers.ts so
server handlers can import it without pulling in client-only modules;
src/config/feeds.ts re-exports for backward compatibility
- Add story tracking Redis key helpers to cache-keys.ts
(story:track:v1, story:sources:v1, story:peak:v1, digest:accumulator:v1)
- Export SEVERITY_SCORES from _classifier.ts for server-side score math
- Add upstashPipeline() to redis.ts for arbitrary batched Redis writes
- Add importanceScore/corroborationCount/storyPhase fields to proto,
generated TS, and src/types NewsItem
- Add StoryMeta message and StoryPhase enum to proto
- In list-feed-digest.ts:
- Build corroboration map across full corpus BEFORE per-category truncation
- Compute importanceScore (severity 40% + tier 20% + corroboration 30%
+ recency 10%) per item
- Sort by importanceScore desc before truncating at MAX_ITEMS_PER_CATEGORY
- Write story:track / story:sources / story:peak / digest:accumulator
to Redis in 80-story pipeline batches after each digest build
Score gate in notification-relay.cjs follows in the next PR (shadow mode,
behind IMPORTANCE_SCORE_LIVE flag). RELAY_GATES_READY removal of
/api/notify comes after 48h shadow comparison confirms parity.
* fix(scoring): add storyPhase field + regenerate proto types
- Add storyPhase to ParsedItem and toProtoItem (defaults UNSPECIFIED)
- Regenerate service_server.ts: required fields, StoryPhase type relocated
- Regenerate service_client.ts and OpenAPI docs from buf generate
- Fix typecheck:api error on missing required storyPhase in NewsItem
* fix(scoring): address all code review findings from PR #2604
P1:
- await writeStoryTracking instead of fire-and-forget to prevent
silent data loss on edge isolate teardown
- remove duplicate upstashPipeline; use existing runRedisPipeline
- strip non-https links before Redis write (XSS prevention)
- implement storyPhase read path: HGETALL batch + computePhase()
so BREAKING/DEVELOPING/SUSTAINED/FADING badges are now live
P2/P3:
- extend STORY_TTL 48h → 7 days (sustained stories no longer reset)
- extract SCORE_WEIGHTS named constants with rationale comment
- move SEVERITY_SCORES out of _classifier.ts into list-feed-digest.ts
- add normalizeTitle comment referencing todo #102
- pre-compute title hashes once, share between phase read + write
* fix(scoring): correct enrichment-before-scoring and write-before-read ordering
Two sequencing bugs:
1. enrichWithAiCache ran after truncation (post-slice), so items whose
threat level was upgraded by the LLM cache could have already been
cut from the top-20, and downgraded items kept inflated scores.
Fix: enrich ALL items from the full corpus before scoring, so
importanceScore always uses the final post-LLM classification level.
2. Phase HGETALL read happened before writeStoryTracking, meaning
first-time stories had no Redis entry and always returned UNSPECIFIED
instead of BREAKING, and all existing stories lagged one cycle behind.
Fix: write tracking first, then read back for phase assignment.
|
||
|
Fayez Bast
|
b2bae30bd8 |
Add climate news seed and ListClimateNews RPC (#2532)
* Add climate news seed and ListClimateNews RPC * Wire climate news into bootstrap and fix generated climate stubs * fix(climate): align seed health interval and parse Atom entries per feed * fix(climate-news): TTL 90min, retry timer on failure, named cache key constant - CACHE_TTL: 1800 to 5400 (90min = 3x 30-min relay interval, gold standard) - ais-relay: add 20-min retry timer on subprocess failure; clear on success - cache-keys.ts: export CLIMATE_NEWS_KEY named constant - list-climate-news.ts: import CLIMATE_NEWS_KEY instead of hard-coding string --------- Co-authored-by: Elie Habib <elie.habib@gmail.com> |
||
|
Fayez Bast
|
bb4f8dcb12 |
feat(climate): add WMO normals seeding and CO2 monitoring (#2531)
* feat(climate): add WMO normals seeding and CO2 monitoring * fix(climate): skip missing normals per-zone and align anomaly tooltip copy * fix(climate): remove normals from bootstrap and harden health/cache key wiring * feat(climate): version anomaly cache to v2, harden seed freshness, and align CO2/normal baselines |
||
|
Elie Habib
|
ae4010a795 |
Revert "feat(climate): add climate disasters seed + ListClimateDisasters RPC with snake_case Redis payload (#2535)" (#2544)
This reverts commit
|
||
|
Fayez Bast
|
e2dea9440d | feat(climate): add climate disasters seed + ListClimateDisasters RPC with snake_case Redis payload (#2535) |