mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
docs(resilience): PR 5.1 — sanctions construct audit (designated-party domicile question) (#3375)
* docs(resilience): PR 5.1 — sanctions construct audit (designated-party domicile question) PR 5.1 of cohort-audit plan 2026-04-24-002. Stacked on PR 5.3 (#3374) so the known-limitations.md section append is additive. Read-only static audit of scoreTradeSanctions + the sanctions:country-counts:v1 seed — framed around the Codex-reformulated construct question: should designated-party domicile count penalize resilience? Findings 1. The count is "OFAC-designated-party domicile locations," NOT "sanctions against this country." Seeder (`scripts/seed-sanctions- pressure.mjs:85-93`) parses OFAC Advanced XML SDN + Consolidated, extracts each designated party's Locations, and increments `map[countryCode]` by 1 for every location country on that party. 2. The count conflates three semantically distinct categories a resilience construct might treat differently: (a) Country-level sanction target (NK SDN listings) — correct penalty (b) Domiciled sanctioned entity (RU bank in Moscow, post-2022) — debatable, country hosts the actor (c) Transit / shell entity (UAE trading co listed under SDGT for Iran evasion; CY SPV for a Russian oligarch) — country is NOT the target, but takes the penalty 3. Observed GCC cohort impact: AE scores 54 vs KW/QA 82. The −28 gap is almost entirely driven by category (c) listings — AE is a financial hub where sanctioned parties incorporate shells. 4. Three options documented for the construct decision (NOT decided in this PR): - Option 1: Keep flat count (status quo, defensible via secondary- sanctions / FATF argument) - Option 2: Program-weighted count — weight DPRK/IRAN/SYRIA/etc. at 1.0, SDGT/SDNTK/CYBER/etc. at 0.3-0.5. Recommended; seeder already captures `programs` per entry — data is there, scorer just doesn't read it. - Option 3: Transit-hub exclusion list (AE, SG, HK, CY, VG, KY) — brittle + normative, not recommended 5. Recommendation documented: Option 2. Implementation deferred to a separate methodology-decision PR (outside auto-mode authority). Shipped - `docs/methodology/known-limitations.md` — new section extending the file: "tradeSanctions — designated-party domicile construct question." Covers what the count represents, the three categories with examples, observed GCC impact, three options w/ trade-offs, recommendation, follow-up audit list (entity-sample gated on API-key access), and file references. - `tests/resilience-sanctions-field-mapping.test.mts` (new) — 10 regression-guard tests pinning CURRENT behavior: 1-6. normalizeSanctionCount piecewise anchors: count=0→100, 1→90, 10→75, 50→50, 200→25, 500→≤1 7. Monotonicity: strictly decreasing across the ramp 8. Country absent from map defaults to count=0 → score 100 (intentional "no designated parties here" semantics) 9. Seed outage (raw=null) → null score slot, NOT imputed (protects against silent data-outage scoring) 10. Construct anchor: count=1 is exactly 10 points below count=0 (pins the "first listing drops 10" design choice) Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail - `npm run test:data` — 6721 pass / 0 fail - `npm run typecheck` / `typecheck:api` — green - `npm run lint` / `lint:md` — clean * fix(resilience): PR 5.1 review — tighten count=500 assertion; clarify weightedBlend weights Addresses 2 P2 Greptile findings on #3375: 1. Tighten count=500 assertion. Was `<= 1` with a comment stating the exact value is 0. That loose bound silently tolerates roundScore / boundary drift that would be the very signal this regression guard exists to catch. Changed to strict equality `=== 0`. 2. Clarify the "zero weight" comment on the sanctions-only harness. The other slots DO contribute their declared weights (0.15 + 0.15 + 0.25 = 0.55) to weightedBlend's `totalWeight` denominator — only `availableWeight` (the score-computation denominator) drops to 0.45 because their score is null. The previous comment elided this distinction and could mislead a reader into thinking the null slots contributed nothing at all. Expanded to state exactly how `coverage` and `score` each behave. Verified - `npx tsx --test tests/resilience-sanctions-field-mapping.test.mts` — 10 pass / 0 fail (count=500 now pins the exact 0 floor)
This commit is contained in:
149
tests/resilience-sanctions-field-mapping.test.mts
Normal file
149
tests/resilience-sanctions-field-mapping.test.mts
Normal file
@@ -0,0 +1,149 @@
|
||||
// Regression guard for scoreTradeSanctions's normalizeSanctionCount
|
||||
// piecewise anchors and field-mapping contract.
|
||||
//
|
||||
// Context. PR 5.1 of plan 2026-04-24-002 (see
|
||||
// `docs/methodology/known-limitations.md#tradesanctions-designated-party-domicile-construct-question`)
|
||||
// documents the construct-ambiguity of counting OFAC-designated-party
|
||||
// domicile locations as a resilience signal. The audit proposes three
|
||||
// options for handling the transit-hub-shell-entity case but
|
||||
// intentionally does NOT implement a scoring change. This test file
|
||||
// pins the CURRENT scorer behavior so that a future methodology
|
||||
// decision (Option 2 = program-weighted count; Option 3 = transit-hub
|
||||
// exclusion; or status quo) updates these tests explicitly.
|
||||
//
|
||||
// Pinning protects against silent scorer refactors: if someone swaps
|
||||
// the piecewise scale, flips the imputation path, or changes how the
|
||||
// seed-outage null branch interacts with weightedBlend, this file
|
||||
// fails before the scoring change propagates to a live publication.
|
||||
|
||||
import assert from 'node:assert/strict';
|
||||
import { describe, it } from 'node:test';
|
||||
|
||||
import {
|
||||
scoreTradeSanctions,
|
||||
type ResilienceSeedReader,
|
||||
} from '../server/worldmonitor/resilience/v1/_dimension-scorers.ts';
|
||||
|
||||
const TEST_ISO2 = 'XX';
|
||||
|
||||
// Minimal synthetic reader: only the sanctions key is populated, so the
|
||||
// scorer's other slots (restrictions, barriers, tariff) drop to null
|
||||
// and contribute zero weight. Isolates the sanctions slot math.
|
||||
function sanctionsOnlyReader(sanctionsCount: number | null): ResilienceSeedReader {
|
||||
return async (key: string) => {
|
||||
if (key === 'sanctions:country-counts:v1') {
|
||||
return sanctionsCount == null ? null : { [TEST_ISO2]: sanctionsCount };
|
||||
}
|
||||
return null;
|
||||
};
|
||||
}
|
||||
|
||||
describe('normalizeSanctionCount — piecewise anchors pinned', () => {
|
||||
// The scorer's piecewise scale (see _dimension-scorers.ts line 535):
|
||||
// count=0 → 100
|
||||
// count=1-10 → 90..75 (linear)
|
||||
// count=11-50 → 75..50 (linear)
|
||||
// count=51-200 → 50..25 (linear)
|
||||
// count=201+ → 25..0 (linear at 0.1/step, clamped 0)
|
||||
//
|
||||
// The tests drive scoreTradeSanctions end-to-end with an otherwise-
|
||||
// empty reader so the sanctions slot is the only one contributing a
|
||||
// non-null score to the weightedBlend. Note the OTHER slots still
|
||||
// contribute their declared weights (restrictions 0.15, barriers
|
||||
// 0.15, tariff 0.25) to weightedBlend's `totalWeight` denominator —
|
||||
// they just don't contribute to `availableWeight` (the score-
|
||||
// computation denominator) because their score is null. So the
|
||||
// surfaced `coverage` value reflects the 0.45 sanctions weight over
|
||||
// the full 1.0 totalWeight; the surfaced `score` reflects the
|
||||
// sanctions-slot score alone (since it's the only non-null input).
|
||||
|
||||
it('count=0 anchors at score 100 (no designated parties)', async () => {
|
||||
const result = await scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(0));
|
||||
assert.equal(result.score, 100, `expected 100 at count=0, got ${result.score}`);
|
||||
});
|
||||
|
||||
it('count=1 anchors at score 90 (first listing drops 10 points)', async () => {
|
||||
const result = await scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(1));
|
||||
assert.equal(result.score, 90, `expected 90 at count=1, got ${result.score}`);
|
||||
});
|
||||
|
||||
it('count=10 anchors at score 75 (end of the 1-10 ramp)', async () => {
|
||||
const result = await scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(10));
|
||||
assert.equal(result.score, 75, `expected 75 at count=10, got ${result.score}`);
|
||||
});
|
||||
|
||||
it('count=50 anchors at score 50 (end of the 11-50 ramp)', async () => {
|
||||
const result = await scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(50));
|
||||
assert.equal(result.score, 50, `expected 50 at count=50, got ${result.score}`);
|
||||
});
|
||||
|
||||
it('count=200 anchors at score 25 (end of the 51-200 ramp)', async () => {
|
||||
const result = await scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(200));
|
||||
assert.equal(result.score, 25, `expected 25 at count=200, got ${result.score}`);
|
||||
});
|
||||
|
||||
it('count=500 anchors at score 0 (high-count tail clamped to floor)', async () => {
|
||||
const result = await scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(500));
|
||||
// At count=500: 25 - (500-200)*0.1 = 25 - 30 = -5 → clamped to 0
|
||||
// via `roundScore` which clamps to [0, 100]. Equality assertion
|
||||
// (not <= 1) so a future roundScore / boundary change that nudges
|
||||
// the result off 0 breaks the test loudly instead of silently.
|
||||
assert.equal(result.score, 0,
|
||||
`expected exactly 0 at count=500 (heavily-sanctioned state; clamped from -5); got ${result.score}`);
|
||||
});
|
||||
|
||||
it('monotonic: more designated parties → strictly lower score', async () => {
|
||||
const scores = await Promise.all([0, 1, 10, 50, 200, 500].map(
|
||||
(n) => scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(n)),
|
||||
));
|
||||
for (let i = 1; i < scores.length; i++) {
|
||||
assert.ok(scores[i].score < scores[i - 1].score,
|
||||
`score must strictly decrease with count; got [${scores.map((s) => s.score).join(', ')}]`);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('scoreTradeSanctions — field-mapping + outage semantics', () => {
|
||||
it('country absent from sanctions map defaults to count=0 (score 100)', async () => {
|
||||
// The map is ISO2 → count. A country NOT in the map is semantically
|
||||
// "no designated parties located here" — NOT "data missing". The
|
||||
// scorer reads `sanctionsCounts[countryCode] ?? 0` (line 1070).
|
||||
const reader: ResilienceSeedReader = async (key) => {
|
||||
if (key === 'sanctions:country-counts:v1') {
|
||||
return { US: 100, RU: 800 }; // our test country XX is NOT in this map
|
||||
}
|
||||
return null;
|
||||
};
|
||||
const result = await scoreTradeSanctions(TEST_ISO2, reader);
|
||||
assert.equal(result.score, 100,
|
||||
`absent-from-map must score 100 (count=0 semantics); got ${result.score}`);
|
||||
});
|
||||
|
||||
it('sanctions seed outage (raw=null) contributes null score slot — NOT imputed', async () => {
|
||||
// When the seed key is entirely absent (not just the country key),
|
||||
// `sanctionsRaw == null` and the slot goes to { score: null, weight: 0.45 }
|
||||
// (line 1082-1083 of _dimension-scorers.ts). This is an intentional
|
||||
// fail-null behavior: we must NOT impute a score on seed outage,
|
||||
// because imputing would mask the outage. The other slots also drop
|
||||
// to null (nothing in our synthetic reader), so weightedBlend returns
|
||||
// coverage=0 — a clean zero-signal state that propagates as low
|
||||
// confidence at the dim level.
|
||||
const reader: ResilienceSeedReader = async () => null;
|
||||
const result = await scoreTradeSanctions(TEST_ISO2, reader);
|
||||
assert.equal(result.coverage, 0,
|
||||
`full-outage must produce coverage=0 (no impute-as-if-clean); got ${result.coverage}`);
|
||||
});
|
||||
|
||||
it('construct-document anchor: count=1 differs from count=0 by exactly 10 points', async () => {
|
||||
// Pins the "first designated party drops the score by 10" design
|
||||
// choice. A future methodology PR that decides Option 2 (program-
|
||||
// weighted) or Option 3 (transit-hub exclusion) will necessarily
|
||||
// update this anchor if the weight-1 semantics change.
|
||||
const [zero, one] = await Promise.all([
|
||||
scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(0)),
|
||||
scoreTradeSanctions(TEST_ISO2, sanctionsOnlyReader(1)),
|
||||
]);
|
||||
assert.equal(zero.score - one.score, 10,
|
||||
`count=1 must be exactly 10 points below count=0; got ${zero.score - one.score}`);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user