Files
worldmonitor/tests/signal-quality.test.mts
Elie Habib 0989f99ae3 feat(insights): ISQ deterministic story ranking + theater pipeline fixes (#2434)
* feat(insights): ISQ deterministic story ranking + theater pipeline fixes

Replace keyword-based getImportanceScore() with ISQ (Investment Signal
Quality), a deterministic 4-dimension scorer using CII scores, focal
point correlation, velocity, and source confidence — no browser ML
dependency.

Also fixes 4 bugs in the theater posture pipeline that caused Iran and
other theater-attributed countries to silently disappear from AI Insights
focal points:

- coordsToCountry() → coordsToCountryWithFallback() so vessels/flights
  over international waters (Persian Gulf, Taiwan Strait) are attributed
  to their theater country instead of returning XX
- getCachedPosture() is now the primary source for theater postures in
  both updateFromServer and updateFromClient, removing the timing-
  dependent guard that skipped ingestion when client flights hadn't
  loaded yet
- Removed hasFlight/hasVessel deduplication guards from
  ingestTheaterPostures that prevented re-ingestion when a misattributed
  signal existed
- Both AI Insights and AI Strategic Posture now consume the same
  getCachedPosture() source, eliminating dual-pipeline divergence

ISQ details:
- New src/utils/signal-quality.ts: normalizeThreatLevel(), computeISQ()
  with 4 weight profiles, handles freeform server threatLevel strings
  (elevated, moderate) not in the TypeScript union
- Focal context passed as functions not singletons to avoid stale-state
  timing bugs; ranking happens AFTER focal+CII refresh
- Signal-backed focal points only (signalCount > 0 || active_strike)
  used for expectation gap — matches renderFocalPoints() filter
- isFocalDataAvailableFn tri-state prevents false novelty bonus when
  analyze() runs with empty clusters
- Server path re-sorts stories by ISQ before sentiment classification
  (positional index alignment preserved via shallow copy)
- ML DETECTED section gated behind localStorage debug flag
- 23 unit tests covering normalization, tri-state gap, tier bounds, all
  weight profiles

* fix(isq): theater posture double-count and server velocity omission

Two bugs surfaced in PR review:

1. ingestTheaterPostures() was not idempotent — it pushed new
   military_flight/vessel signals on every InsightsPanel refresh
   without clearing the previous ones, inflating focal scores and
   CII on every rerender. Fix: track theater-added signals in a
   private array and remove them at the start of each call.

2. Server-path ISQ sort omitted velocity, causing spike/elevated
   stories to fall back to normal timeliness baseline and diverge
   from client-path ranking. Fix: pass a.velocity/b.velocity into
   computeISQ; make velocity.trend optional in SignalQualityInput
   since ServerInsightStory does not carry trend.

* fix(isq): three review findings — theater dedup, actor attribution, brief context

P1 — Within-cycle double-count in ingestTheaterPostures:
coordsToCountryWithFallback already attributes real flights/vessels
to theater countries (IR, TW, etc). Theater posture was adding a
second military_flight/vessel signal on top. Fix: collect active
theater codes first, remove any existing military_flight/vessel
signals for those countries before adding the theater summary.
Theater posture is now the single authoritative source for its
target theaters.

P2 — Wrong country assigned via shared-actor keywords:
extractISQInput took the first country entity from title extraction,
but keyword matches (confidence 0.7) expand shared terms like
"hezbollah" to both IR and IL, and "hamas" to both IL and QA.
Fix: only accept alias-matched country entities (direct country name
mentioned, confidence >= 0.85) for ISQ country attribution. Titles
with only actor-keyword matches get countryCode null (neutral gap),
which is the correct conservative behavior.

P3 — World Brief prompt did not see cached theater posture:
getTheaterPostureContext() returned early if lastMilitaryFlights was
empty, so in the "flights not loaded yet, cached posture exists" case
the brief had no theater context while scoring did. Fix: use same
getCachedPosture() ?? flights fallback pattern as the scoring path.

* fix(signal-aggregator): type-specific theater dedup to prevent undercount

Previous fix cleared both military_flight and military_vessel signals
for any active theater country, but the re-add is conditional per type
(only when totalAircraft > 0 / totalVessels > 0). An aircraft-only
theater posture would drop real vessel signals for that country with
no replacement.

Fix: track activeFlightCodes and activeVesselCodes separately, then
filter by matching country+type pair. Each signal type is only removed
when theater posture will actually replace it.

* fix(insights): cached postures empty-array must not suppress live-flight fallback

getCachedPosture()?.postures ?? fallback only falls back on null/undefined.
An empty array [] from a backend that returned no theaters writes to
localStorage, rehydrates as fresh, and blocks the live-flight path
indefinitely.

Fix all three call sites (getTheaterPostureContext, updateInsights x2):
guard on .length so an empty cached array still triggers the fallback
to getTheaterPostureSummaries(lastMilitaryFlights).
2026-03-28 19:52:59 +04:00

111 lines
4.8 KiB
TypeScript

import assert from 'node:assert/strict';
import { describe, it } from 'node:test';
import { computeISQ, normalizeThreatLevel, type SignalQualityInput } from '../src/utils/signal-quality.ts';
const noFocal = () => null;
const focalAvailable = false;
const noFocalFn = () => null;
const noFocalReady = () => false;
const focalReady = () => true;
function isq(input: SignalQualityInput, focalFn = noFocalFn, ciiFn: (c: string) => number | null = () => null, ready = noFocalReady) {
return computeISQ(input, focalFn, ciiFn, ready);
}
describe('normalizeThreatLevel', () => {
it('maps critical to 1.0', () => assert.equal(normalizeThreatLevel('critical'), 1.0));
it('maps high to 0.75', () => assert.equal(normalizeThreatLevel('high'), 0.75));
it('maps elevated to 0.55', () => assert.equal(normalizeThreatLevel('elevated'), 0.55));
it('maps moderate to 0.4', () => assert.equal(normalizeThreatLevel('moderate'), 0.4));
it('maps medium to 0.4', () => assert.equal(normalizeThreatLevel('medium'), 0.4));
it('maps low to 0.2', () => assert.equal(normalizeThreatLevel('low'), 0.2));
it('maps info to 0.1', () => assert.equal(normalizeThreatLevel('info'), 0.1));
it('maps unknown to 0.3', () => assert.equal(normalizeThreatLevel('unknown'), 0.3));
it('maps undefined to 0.3', () => assert.equal(normalizeThreatLevel(undefined), 0.3));
it('is case-insensitive', () => assert.equal(normalizeThreatLevel('CRITICAL'), 1.0));
});
describe('computeISQ — composite bounds', () => {
it('composite is always in [0, 1]', () => {
const inputs: SignalQualityInput[] = [
{ sourceCount: 0, isAlert: false },
{ sourceCount: 5, isAlert: true, threatLevel: 'critical', velocity: { sourcesPerHour: 10, level: 'spike', trend: 'rising' } },
{ sourceCount: 1, isAlert: false, threatLevel: 'info', countryCode: 'US' },
];
for (const input of inputs) {
const result = isq(input);
assert.ok(result.composite >= 0 && result.composite <= 1, `composite out of range: ${result.composite}`);
}
});
});
describe('computeISQ — no-country gap', () => {
it('gap is 0.5 when no countryCode', () => {
const result = isq({ sourceCount: 1, isAlert: false });
assert.equal(result.expectationGap, 0.5);
});
it('gap is 0.5 when null countryCode', () => {
const result = isq({ sourceCount: 1, isAlert: false, countryCode: null });
assert.equal(result.expectationGap, 0.5);
});
});
describe('computeISQ — expectation gap tri-state', () => {
it('gap is 0.4 when country is a signal-backed focal point (present)', () => {
const focalFn = () => ({ focalScore: 60, urgency: 'elevated' });
const result = isq({ sourceCount: 2, isAlert: false, countryCode: 'IR' }, focalFn);
assert.equal(result.expectationGap, 0.4);
});
it('gap is 0.8 when focal data available but country absent (novel)', () => {
const result = isq({ sourceCount: 2, isAlert: false, countryCode: 'DE' }, noFocalFn, () => null, focalReady);
assert.equal(result.expectationGap, 0.8);
});
it('gap is 0.5 when focal data unavailable (neutral)', () => {
const result = isq({ sourceCount: 2, isAlert: false, countryCode: 'DE' }, noFocalFn, () => null, noFocalReady);
assert.equal(result.expectationGap, 0.5);
});
});
describe('computeISQ — CII not warmed up', () => {
it('intensity falls back to threatLevel only when ciiScoreFn returns null', () => {
const result = isq({ sourceCount: 1, isAlert: false, threatLevel: 'high', countryCode: 'IR' }, noFocalFn, () => null);
assert.equal(result.intensity, 0.75);
});
});
describe('computeISQ — tiers', () => {
it('strong tier for high-confidence + high-intensity story', () => {
const focalFn = () => ({ focalScore: 90, urgency: 'critical' });
const result = isq(
{ sourceCount: 3, isAlert: true, threatLevel: 'critical', velocity: { sourcesPerHour: 5, level: 'spike', trend: 'rising' }, countryCode: 'IR' },
focalFn,
() => 85,
);
assert.equal(result.tier, 'strong');
});
it('weak tier for low-signal story (single source, info threat)', () => {
const result = isq({ sourceCount: 1, isAlert: false, threatLevel: 'info' });
assert.ok(result.tier === 'weak' || result.tier === 'noise', `expected weak/noise, got ${result.tier}`);
assert.ok(result.composite < 0.5);
});
});
describe('computeISQ — weight profiles sum to 1', () => {
const WEIGHTS: Record<string, [number, number, number, number]> = {
default: [0.35, 0.30, 0.20, 0.15],
risk: [0.45, 0.25, 0.20, 0.10],
macro: [0.25, 0.40, 0.20, 0.15],
shortTerm: [0.30, 0.25, 0.20, 0.25],
};
for (const [name, w] of Object.entries(WEIGHTS)) {
it(`${name} weights sum to 1.0`, () => {
const sum = w.reduce((a, b) => a + b, 0);
assert.ok(Math.abs(sum - 1.0) < 1e-10, `${name} weights sum to ${sum}`);
});
}
});