mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-26 01:24:59 +02:00
75b3b0026b
* feat(resilience): dimension freshness propagation (T1.5 propagation pass) Ships the Phase 1 T1.5 propagation pass of the country-resilience reference-grade upgrade plan. PR #2947 shipped the staleness classifier foundation (classifyStaleness, cadence taxonomy, three staleness levels) and explicitly deferred the dimension-level propagation. This PR consumes the classifier and surfaces per dimension freshness on the ResilienceDimension response. What this PR commits - Proto: new DimensionFreshness message + `freshness` field on ResilienceDimension (last_observed_at_ms, staleness string). - New module server/worldmonitor/resilience/v1/_dimension-freshness.ts that reads seed-meta values for every sourceKey in INDICATOR_REGISTRY and aggregates the worst staleness + oldest fetchedAt across the constituent indicators of each dimension. - scoreAllDimensions decorates each dimension score with its freshness result before returning. The 13 dimension scorer function bodies are untouched: aggregation is a decoration pass at the caller level so this PR stays mechanical. - Response builder: _shared.ts buildDimensionList propagates the freshness field to the proto output. - Tests: 10 classifyDimensionFreshness + readFreshnessMap cases in a new test file + response-shape case on the release-gate test. Aggregation rules - last_observed_at_ms: MIN fetchedAt across the dimension's indicators (oldest signal = most conservative bound). 0 when no signal has ever been observed. - staleness: MAX staleness level across the dimension's indicators (stale > aging > fresh). Empty string when the dimension has no indicators in the registry (defensive path). What is deliberately NOT in this PR - No changes to the 13 individual dimension scorer function bodies. Per-signal freshness inside scorers is a future enhancement. - No widget rendering of the freshness badge (T1.6 full grid, PR 3). - No cache key bump: additive int64/string fields with zero defaults. Verified - make generate clean, new interface in regenerated types - typecheck + typecheck:api clean - tests/resilience-dimension-freshness.test.mts all new cases pass - tests/resilience-*.test.mts full suite pass - test:data clean - lint exits 0 on touched files * fix(resilience): resolve templated sourceKeys to real seed-meta (#2961 P1) Greptile P1 finding on PR #2961: readFreshnessMap() assumed every INDICATOR_REGISTRY sourceKey could be fetched as seed-meta:<sourceKey>, but most entries use placeholder templates like resilience:static:{ISO2}, energy:mix:v1:{ISO2}, and displacement:summary:v1:{year}. Those produce literal lookups like seed-meta:resilience:static:{ISO2} which don't exist in Redis, so the freshness map missed every templated entry and classifyDimensionFreshness marked the affected dimensions stale even with healthy seeds. Most Phase 1 T1.5 freshness badges were broken on arrival. Fix: two-layer resolution in _dimension-freshness.ts. Layer 1 stripTemplateTokens: drop :{placeholder} and :* segments. 'resilience:static:{ISO2}' -> 'resilience:static' 'resilience:static:*' -> 'resilience:static' 'energy:mix:v1:{ISO2}' -> 'energy:mix:v1' 'displacement:summary:v1:{year}' -> 'displacement:summary:v1' Layer 2 stripTrailingVersion: strip trailing :v\d+, mirroring writeExtraKeyWithMeta + runSeed() in scripts/_seed-utils.mjs which never persist the trailing version in seed-meta keys. Handles cyber:threats:v2, infra:outages:v1, unrest:events:v1, conflict:ucdp-events:v1, sanctions:country-counts:v1, and the displacement v1 case above. Layer 3 SOURCE_KEY_META_OVERRIDES: explicit table for drift cases where the two strips still do not match the real seed-meta key. Verified against api/seed-health.js, api/health.js, and scripts/seed-*. Drift cases covered: economic:imf:macro -> economic:imf-macro economic:bis:eer -> economic:bis economic:energy:v1:all -> economic:energy-prices energy:mix -> economic:owid-energy-mix energy:gas-storage -> energy:gas-storage-countries news:threat:summary -> news:threat-summary intelligence:social:reddit -> intelligence:social-reddit readFreshnessMap now deduplicates reads by resolved meta key (so the 15+ resilience:static indicators share one Redis read) and projects per-meta-key results back onto per-sourceKey map entries so classifyDimensionFreshness can keep its existing interface. Regression coverage: - stripTemplateTokens cases for {ISO2}, {year}, and *. - stripTrailingVersion cases for :v1 / :v2 suffixes. - Embedded :v1 carve-out (trade:restrictions:v1:tariff-overview:50 stays unchanged because :v1 is not trailing). - Override cases for the seven drift entries. - Integration test that proves every resilience:static:* / {ISO2} registry entry resolves to the same seed-meta and is marked fresh when that one key has a recent fetchedAt. - healthPublicService end-to-end test: classifies fresh when seed-meta:resilience:static is recent (was stale before the fix). - Registry-coverage assertion: every INDICATOR_REGISTRY sourceKey must resolve to a seed-meta key that either lives in api/seed-health.js, api/health.js, or the test's KNOWN_SEEDS_NOT_IN_HEALTH allowlist (which covers the four seeds written by writeExtraKeyWithMeta / runSeed that no health monitor tracks yet: trade:restrictions, trade:barriers, sanctions:country-counts, economic:energy-prices). Fails loudly if a future registry entry introduces an unknown sourceKey. Note on P1 #2 (scoreCurrencyExternal absence-branch delete): that is PR #2964's scope (T1.7 source-failure wiring), not #2961 (T1.5 propagation pass). #2961 never claimed to delete the fallback branch; no test in this branch expects the new IMPUTE.bisEer fallback. The reviewer conflated the two stacked PRs. #2964 owns the delete.
1 line
5.3 KiB
JSON
1 line
5.3 KiB
JSON
{"components":{"schemas":{"DimensionFreshness":{"properties":{"lastObservedAtMs":{"description":"Unix milliseconds when the oldest constituent signal in this\n dimension was last observed (min fetchedAt across INDICATOR_REGISTRY\n entries for this dimension). 0 when no signal has ever been\n observed.","format":"int64","type":"string"},"staleness":{"description":"Worst staleness level across the dimension's constituent signals,\n classified by classifyStaleness against each signal's cadence.\n One of: \"fresh\", \"aging\", \"stale\". Empty string when no signals.","type":"string"}},"type":"object"},"Error":{"description":"Error is returned when a handler encounters an error. It contains a simple error message that the developer can customize.","properties":{"message":{"description":"Error message (e.g., 'user not found', 'database connection failed')","type":"string"}},"type":"object"},"FieldViolation":{"description":"FieldViolation describes a single validation error for a specific field.","properties":{"description":{"description":"Human-readable description of the validation violation (e.g., 'must be a valid email address', 'required field missing')","type":"string"},"field":{"description":"The field path that failed validation (e.g., 'user.email' for nested fields). For header validation, this will be the header name (e.g., 'X-API-Key')","type":"string"}},"required":["field","description"],"type":"object"},"GetResilienceRankingRequest":{"type":"object"},"GetResilienceRankingResponse":{"properties":{"greyedOut":{"items":{"$ref":"#/components/schemas/ResilienceRankingItem"},"type":"array"},"items":{"items":{"$ref":"#/components/schemas/ResilienceRankingItem"},"type":"array"}},"type":"object"},"GetResilienceScoreRequest":{"properties":{"countryCode":{"type":"string"}},"type":"object"},"GetResilienceScoreResponse":{"properties":{"baselineScore":{"format":"double","type":"number"},"change30d":{"format":"double","type":"number"},"countryCode":{"type":"string"},"dataVersion":{"type":"string"},"domains":{"items":{"$ref":"#/components/schemas/ResilienceDomain"},"type":"array"},"imputationShare":{"format":"double","type":"number"},"level":{"type":"string"},"lowConfidence":{"type":"boolean"},"overallScore":{"format":"double","type":"number"},"scoreInterval":{"$ref":"#/components/schemas/ScoreInterval"},"stressFactor":{"format":"double","type":"number"},"stressScore":{"format":"double","type":"number"},"trend":{"type":"string"}},"type":"object"},"ResilienceDimension":{"properties":{"coverage":{"format":"double","type":"number"},"freshness":{"$ref":"#/components/schemas/DimensionFreshness"},"id":{"type":"string"},"imputationClass":{"description":"Four-class imputation taxonomy (Phase 1 T1.7). Empty string when the\n dimension has any observed data. One of: \"stable-absence\", \"unmonitored\",\n \"source-failure\", \"not-applicable\". See docs/methodology/country-resilience-index.mdx.","type":"string"},"imputedWeight":{"format":"double","type":"number"},"observedWeight":{"format":"double","type":"number"},"score":{"format":"double","type":"number"}},"type":"object"},"ResilienceDomain":{"properties":{"dimensions":{"items":{"$ref":"#/components/schemas/ResilienceDimension"},"type":"array"},"id":{"type":"string"},"score":{"format":"double","type":"number"},"weight":{"format":"double","type":"number"}},"type":"object"},"ResilienceRankingItem":{"properties":{"countryCode":{"type":"string"},"level":{"type":"string"},"lowConfidence":{"type":"boolean"},"overallCoverage":{"format":"double","type":"number"},"overallScore":{"format":"double","type":"number"},"rankStable":{"type":"boolean"}},"type":"object"},"ScoreInterval":{"properties":{"p05":{"format":"double","type":"number"},"p95":{"format":"double","type":"number"}},"type":"object"},"ValidationError":{"description":"ValidationError is returned when request validation fails. It contains a list of field violations describing what went wrong.","properties":{"violations":{"description":"List of validation violations","items":{"$ref":"#/components/schemas/FieldViolation"},"type":"array"}},"required":["violations"],"type":"object"}}},"info":{"title":"ResilienceService API","version":"1.0.0"},"openapi":"3.1.0","paths":{"/api/resilience/v1/get-resilience-ranking":{"get":{"operationId":"GetResilienceRanking","responses":{"200":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/GetResilienceRankingResponse"}}},"description":"Successful response"},"400":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ValidationError"}}},"description":"Validation error"},"default":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}},"description":"Error response"}},"summary":"GetResilienceRanking","tags":["ResilienceService"]}},"/api/resilience/v1/get-resilience-score":{"get":{"operationId":"GetResilienceScore","parameters":[{"in":"query","name":"countryCode","required":true,"schema":{"type":"string"}}],"responses":{"200":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/GetResilienceScoreResponse"}}},"description":"Successful response"},"400":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/ValidationError"}}},"description":"Validation error"},"default":{"content":{"application/json":{"schema":{"$ref":"#/components/schemas/Error"}}},"description":"Error response"}},"summary":"GetResilienceScore","tags":["ResilienceService"]}}}} |