Files
worldmonitor/tests/gold-cb-reserves-seed.test.mjs
Elie Habib 30ddad28d7 fix(seeds): upstream API drift — SPDR XLSX + IMF IRFCL + IMF-External BX/BM drop (#3076)
* fix(seeds): gold-etf XLSX migration, IRFCL dataflow, imf-external BX/BM drop

Three upstream-drift regressions caught from the market-backup + imf-extended
bundle logs. Root causes validated by live API probes before coding.

1. seed-gold-etf-flows: SPDR /assets/dynamic/GLD/GLD_US_archive_EN.csv now
   silently returns a PDF (Content-Type: application/pdf) — site migrated
   to api.spdrgoldshares.com/api/v1/historical-archive which serves XLSX.
   Swapped the CSV parser for an exceljs-based XLSX parser. Adds
   browser-ish Origin/Referer headers (SPDR swaps payload for PDF
   without them) and a Content-Type guard. Column layout: Date | Closing |
   ... | Tonnes | Total NAV USD.

2. seed-gold-cb-reserves: PR #3038 shipped with IMF.STA/IFS dataflow and
   3-segment key M..<indicator> — both wrong. IFS isn't exposed on
   api.imf.org (HTTP 204). Gold-reserves data lives under IMF.STA/IRFCL
   with 4 dimensions (COUNTRY.INDICATOR.SECTOR.FREQUENCY). Verified live:
   *.IRFCLDT1_IRFCL56_FTO.*.M returns 111 series. Switched to IRFCL +
   IRFCLDT1_IRFCL56_FTO (fine troy ounces) and fallbacks. The
   valueIsOunces flag now matches _FTO suffix (keeps legacy _OZT/OUNCE
   detection for backward compat).

3. seed-imf-external: BX/BM (export/import LEVELS, USD) WEO coverage
   collapsed to ~10 countries in late 2026 — the seeder's >=190-country
   validate floor was failing every run. Dropped BX/BM from fetch + join;
   kept BCA (~209) / TM_RPCH (~189) / TX_RPCH (~190). exportsUsd /
   importsUsd / tradeBalanceUsd fields kept as explicit null so consumers
   see a deliberate gap. validate floor lowered to 180 (BCA∪TM∪TX union).

Tests: 32/32 pass. Rewrote gold-etf tests to use synthetic XLSX fixtures
(exceljs resolved from scripts/package.json since repo root doesn't have
it). Updated imf-external tests for the new indicator set + null BX/BM
contract + 180-country validate threshold.

* fix(mcp): update get_country_macro description after BX/BM drop

Consumer-side catch during PR #3076 validation: the MCP tool description
still promised 'exports, imports, trade balance' fields that the seeder
fix nulls out. LLM consumers would be directed to exportsUsd/importsUsd/
tradeBalanceUsd fields that always return null since seed-imf-external
dropped BX/BM (WEO coverage collapsed to ~10 countries).

Updated description to list only the indicators actually populated
(currentAccountUsd, importVolumePctChg, exportVolumePctChg) with an
explicit note about the null trade-level fields so LLMs don't attempt
to use them.

* fix(gold-cb-reserves): compute real pctOfReserves + add exceljs to root

Follow-up to #3076 review.

1. pctOfReserves was hardcoded to 0 with a "IFS doesn't give us total
   reserves" comment. That was a lazy limitation claim — IMF IRFCL DOES
   expose total official reserve assets as IRFCLDT1_IRFCL65_USD parallel
   to the gold USD series IRFCLDT1_IRFCL56_USD. fetchCbReserves now
   pulls all three indicators (primary FTO tonnage + the two USD series)
   via Promise.allSettled and passes the USD pair to buildReservesPayload
   so it can compute the true gold share per country. Falls back to 0
   only when the denominator is genuinely missing for that country
   (IRFCL coverage: 114 gold_usd, 96 total_usd series; ~15% of holders
   have no matched-month denominator). 3-month lookback window absorbs
   per-country reporting lag.

2. CI fix: tests couldn't find exceljs because scripts/package.json is
   not workspace-linked to the repo root — CI runs `npm ci` at root
   only. Added exceljs@^4.4.0 as a root devDependency. Runtime seeder
   continues to resolve it from scripts/node_modules via Node's upward
   module resolution.

3 new tests cover pct computation, missing-denominator fallback, and
the 3-month lookback window.
2026-04-14 08:19:47 +04:00

161 lines
7.5 KiB
JavaScript

import { describe, it } from 'node:test';
import assert from 'node:assert/strict';
import { latestMonth, monthOffset, buildReservesPayload } from '../scripts/seed-gold-cb-reserves.mjs';
describe('seed-gold-cb-reserves: latestMonth', () => {
it('returns the lexicographically latest YYYY-MM key', () => {
assert.equal(latestMonth({ '2025-12': 1, '2026-01': 2, '2025-11': 3 }), '2026-01');
});
it('returns undefined on empty object', () => {
assert.equal(latestMonth({}), undefined);
});
});
describe('seed-gold-cb-reserves: monthOffset', () => {
it('subtracts 12 months across year boundary', () => {
assert.equal(monthOffset('2026-02', -12), '2025-02');
});
it('subtracts 1 month within year', () => {
assert.equal(monthOffset('2026-04', -1), '2026-03');
});
it('handles January rollback to previous December', () => {
assert.equal(monthOffset('2026-01', -1), '2025-12');
});
});
describe('seed-gold-cb-reserves: buildReservesPayload (ounces indicator)', () => {
// 1 tonne = 32150.7 troy oz. US holdings ~8133 tonnes → 261.5M oz.
const raw = {
USA: { name: 'United States', byMonth: { '2025-01': 261_500_000, '2026-01': 261_500_000 } },
DEU: { name: 'Germany', byMonth: { '2025-01': 108_000_000, '2026-01': 108_000_000 } },
CHN: { name: 'China', byMonth: { '2025-01': 70_000_000, '2026-01': 74_000_000 } },
TUR: { name: 'Turkey', byMonth: { '2025-01': 18_000_000, '2026-01': 20_000_000 } },
// Two sellers
UZB: { name: 'Uzbekistan', byMonth: { '2025-01': 12_000_000, '2026-01': 10_000_000 } },
CAN: { name: 'Canada', byMonth: { '2025-01': 5_000_000, '2026-01': 4_000_000 } },
// Aggregate that must be filtered out
EU: { name: 'European Union', byMonth: { '2025-01': 500_000_000, '2026-01': 500_000_000 } },
};
it('drops aggregate codes and sorts holders descending by tonnes', () => {
const payload = buildReservesPayload(raw, 'RAFAGOLDV_OZT');
assert.ok(payload !== null);
assert.equal(payload.asOfMonth, '2026-01');
assert.equal(payload.valueIsOunces, true);
assert.ok(!payload.topHolders.some(h => h.iso3 === 'EU'), 'EU aggregate must be filtered');
assert.equal(payload.topHolders[0].iso3, 'USA');
assert.equal(payload.topHolders[1].iso3, 'DEU');
assert.equal(payload.topHolders[2].iso3, 'CHN');
});
it('computes 12-month tonnage deltas correctly', () => {
const payload = buildReservesPayload(raw, 'RAFAGOLDV_OZT');
// China: +4M oz → +124.4 tonnes
const cn = payload.topBuyers12m.find(m => m.iso3 === 'CHN');
assert.ok(cn);
assert.ok(Math.abs(cn.deltaTonnes12m - 124.4) < 0.5, `got ${cn.deltaTonnes12m}`);
// Turkey: +2M oz → +62.2 tonnes
const tr = payload.topBuyers12m.find(m => m.iso3 === 'TUR');
assert.ok(Math.abs(tr.deltaTonnes12m - 62.2) < 0.5);
// Uzbekistan: -2M oz → -62.2 tonnes (seller)
const uz = payload.topSellers12m.find(m => m.iso3 === 'UZB');
assert.ok(uz);
assert.ok(uz.deltaTonnes12m < 0);
});
it('returns null when no non-aggregate data exists', () => {
const payload = buildReservesPayload({ EU: { name: 'EU', byMonth: { '2026-01': 500_000_000 } } }, 'RAFAGOLDV_OZT');
assert.equal(payload, null);
});
it('skips countries missing the latest month value', () => {
const partial = {
USA: { name: 'United States', byMonth: { '2025-01': 261_500_000, '2026-01': 261_500_000 } },
DEU: { name: 'Germany', byMonth: { '2025-01': 108_000_000 } }, // no 2026-01
};
const payload = buildReservesPayload(partial, 'RAFAGOLDV_OZT');
assert.ok(payload);
assert.equal(payload.topHolders.length, 1);
assert.equal(payload.topHolders[0].iso3, 'USA');
});
});
describe('seed-gold-cb-reserves: buildReservesPayload (USD indicator)', () => {
it('keeps USD values but marks tonnes as 0 (unknown) and skips deltas', () => {
const raw = {
USA: { name: 'United States', byMonth: { '2025-01': 600_000_000_000, '2026-01': 700_000_000_000 } },
};
const payload = buildReservesPayload(raw, 'RAFAGOLD_USD');
assert.ok(payload);
assert.equal(payload.valueIsOunces, false);
assert.equal(payload.topHolders[0].tonnes, 0); // USD series can't derive tonnes here
// Delta stays zero because USD moves can be price-driven, not buying
assert.equal(payload.topBuyers12m.length, 0);
});
});
describe('seed-gold-cb-reserves: pctOfReserves computation', () => {
it('computes gold share of total reserves when both USD series are supplied', () => {
const raw = {
USA: { name: 'United States', byMonth: { '2026-01': 261_500_000 } },
DEU: { name: 'Germany', byMonth: { '2026-01': 108_000_000 } },
CHN: { name: 'China', byMonth: { '2026-01': 74_000_000 } },
};
// Gold USD value per country (market value, approximate)
const goldUsd = {
USA: { byMonth: { '2026-01': 400_000_000_000 } },
DEU: { byMonth: { '2026-01': 170_000_000_000 } },
CHN: { byMonth: { '2026-01': 115_000_000_000 } },
};
// Total reserve assets per country
const totalUsd = {
USA: { byMonth: { '2026-01': 800_000_000_000 } }, // US: 50% gold share (synthetic)
DEU: { byMonth: { '2026-01': 250_000_000_000 } }, // DE: 68% (synthetic, realistic)
CHN: { byMonth: { '2026-01': 3_300_000_000_000 } }, // CN: ~3.5% (realistic — mostly FX)
};
const payload = buildReservesPayload(raw, 'IRFCLDT1_IRFCL56_FTO', goldUsd, totalUsd);
assert.ok(payload);
const by = Object.fromEntries(payload.topHolders.map(h => [h.iso3, h]));
assert.equal(by.USA.pctOfReserves, 50);
assert.equal(by.DEU.pctOfReserves, 68);
assert.equal(by.CHN.pctOfReserves, 3.48);
});
it('falls back to pctOfReserves=0 when denominator series is missing for a country', () => {
const raw = {
USA: { name: 'United States', byMonth: { '2026-01': 261_500_000 } },
};
// Gold USD present, but total reserves missing for USA
const goldUsd = { USA: { byMonth: { '2026-01': 400_000_000_000 } } };
const totalUsd = {};
const payload = buildReservesPayload(raw, 'IRFCLDT1_IRFCL56_FTO', goldUsd, totalUsd);
assert.equal(payload.topHolders[0].pctOfReserves, 0, 'no denominator → 0');
});
it('accepts a denominator from 1-2 months before asOfMonth (per-country reporting lag)', () => {
// Primary tonnage reports 2026-03. Total reserves only has 2026-02 (1mo lag).
// Should still compute pctOfReserves using the 2026-02 denominator.
const raw = {
USA: { name: 'United States', byMonth: { '2026-03': 261_500_000 } },
};
const goldUsd = { USA: { byMonth: { '2026-03': 400_000_000_000 } } };
const totalUsd = { USA: { byMonth: { '2026-02': 800_000_000_000 } } };
const payload = buildReservesPayload(raw, 'IRFCLDT1_IRFCL56_FTO', goldUsd, totalUsd);
assert.equal(payload.asOfMonth, '2026-03');
assert.equal(payload.topHolders[0].pctOfReserves, 50, '2026-02 total is within the 3-month lookback window');
});
it('rejects a denominator older than 3 months (stale data shouldn\'t contaminate current pct)', () => {
const raw = {
USA: { name: 'United States', byMonth: { '2026-06': 261_500_000 } },
};
const goldUsd = { USA: { byMonth: { '2026-06': 400_000_000_000 } } };
// Total reserves last reported 2026-01 — 5 months before; outside the window.
const totalUsd = { USA: { byMonth: { '2026-01': 800_000_000_000 } } };
const payload = buildReservesPayload(raw, 'IRFCLDT1_IRFCL56_FTO', goldUsd, totalUsd);
assert.equal(payload.topHolders[0].pctOfReserves, 0, 'stale denominator (>3mo) is dropped');
});
});