mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
feat(brief): per-user composer writing brief:{userId}:{issueDate} (Phase 3a) (#3154)
* feat(brief): per-user composer writing brief:{userId}:{issueDate} (Phase 3a)
Phase 3a of docs/plans/2026-04-17-003. Produces the Redis-resident
envelopes that Phases 1 (renderer) and 2 (edge routes) already know
how to serve, so after this ships the end-to-end read path works
with real data.
Files:
- shared/brief-filter.{js,d.ts}: pure helpers. normaliseThreatLevel
maps upstream 'moderate' -> 'medium' (contract pinned the union in
Phase 1). filterTopStories applies sensitivity thresholds and caps
at maxStories. assembleStubbedBriefEnvelope builds a full envelope
with stubbed greeting/lead/threads/signals and runs it through the
renderer's assertBriefEnvelope so no malformed envelope is ever
persisted. issueDateInTz computes per-user local date via Intl
with UTC fallback.
- scripts/seed-brief-composer.mjs: Railway cron. Reads
news:insights:v1 once, fetches enabled alert rules via the
existing /relay/digest-rules endpoint (same set
seed-digest-notifications uses), then for each rule computes the
user's local issue date, filters stories, assembles an envelope,
and SETEX brief:{userId}:{issueDate} with 7-day TTL. Respects
aiDigestEnabled opt-in. Honours SIGTERM. Exits non-zero when >5%
of rules fail so Railway surfaces structural breakage.
- Dockerfile.seed-brief-composer: standalone container. Copies the
minimum set (composer + shared/ contract + renderer validator +
Upstash helper + seed-envelope unwrapper).
- tests/brief-filter.test.mjs: 22 pure-function tests covering
severity normalisation (including 'moderate' alias), sensitivity
thresholds, story cap, empty-title drop, envelope assembly passes
the strict renderer validator, tz-aware date math across +UTC/-UTC
offsets with a bad-timezone fallback.
Out of scope for this PR:
- LLM-generated whyMatters / lead / signals (Phase 3b).
- brief_ready event fan-out to notification-relay (Phase 3c).
- Dashboard panel that consumes /api/latest-brief (Phase 4).
Pre-merge runbook:
1. Create a new Railway service from Dockerfile.seed-brief-composer.
2. Set env vars (UPSTASH_*, CONVEX_URL, RELAY_SHARED_SECRET) — reuse
the values already in the digest service.
3. Add a cron schedule (suggested: hourly at :05 so it lands between
the insights-seeder tick and the digest cron).
4. Verify first run: check service logs for
"[brief-composer] Done: success=X ..." and a reader's
/api/latest-brief should stop returning 'composing' within one
cron cycle.
Tests: 72/72 (22 brief-filter + 30 render + 20 HMAC). Typecheck +
lint clean. Composer script parses with node --check.
* fix(brief): aiDigestEnabled default + per-user rule dedupe
Addresses two fourth-round review findings on PR #3154.
1. aiDigestEnabled default parity (todo 224). Composer was checking
`!rule.aiDigestEnabled`, which skips legacy rules that predate the
optional field. The rest of the codebase defaults it to true
(seed-digest-notifications.mjs:914 uses `!== false`;
notifications-settings.ts:228 uses `?? true`; the Convex setter
defaults to true). Flipped the composer to `=== false` so only an
explicit opt-out skips the brief.
2. Multi-variant last-write-wins (todo 225). alertRules are
(userId, variant)-scoped but the brief key is user-scoped
(brief:{userId}:{issueDate}). Users with the full+finance+tech
variants all enabled would produce three competing writes with a
nondeterministic survivor. Added dedupeRulesByUser() that picks
one rule per user: prefers 'full' variant, then most permissive
sensitivity (all > high > critical), tie-breaking on earliest
updatedAt for stability across input reordering. Logs the
occurrence so we can see how often users have multi-variant
configs.
Also hardened against future regressions:
- Moved env-var guards + main() call behind an isMain() check
(feedback_seed_isMain_guard). Previously, importing the script
from a test would fire process.exit(0) on the
BRIEF_COMPOSER_ENABLED=0 branch and kill the test runner. Tests
now load the file cleanly.
- Exported dedupeRulesByUser so the tests can exercise the selection
logic directly.
- The new tests/brief-composer-rule-dedup.test.mjs includes a
cross-module assertion that seed-digest-notifications.mjs still
reads `rule.aiDigestEnabled !== false`. If the digest cron ever
drifts, this test fails loud — the brief and digest must agree on
who is eligible.
Tests: 83/83 (was 72; +6 dedupe cases + 5 aiDigestEnabled parity
cases). Typecheck + lint clean.
* fix(brief): dedupe order + failure-rate denominator
Addresses two fifth-round review findings on PR #3154.
1. Dedupe was picking a preferred variant BEFORE checking whether it
could actually emit a brief (todo 226). A user with
aiDigestEnabled=false on 'full' but true on 'finance' got skipped
entirely; same for a user with sensitivity='critical' on 'full'
that filters to zero stories while 'finance' has matching content.
Replaced dedupeRulesByUser with groupEligibleRulesByUser: pre-
filters opted-out rules, then returns ALL eligible variants per
user in preference order. The main loop walks candidates and
takes the first one whose story filter produces non-empty content.
Fallback is cheap (story filter is pure) and preserves the 'full'-
first + most-permissive-sensitivity tie-breakers from before.
dedupeRulesByUser is kept as a thin wrapper for the existing tests;
new tests exercise the group+fallback path directly (opt-out +
opt-in sibling, all-opted-out drop, ordering stability).
2. Failure gate denominator drifted from numerator (todo 227). After
dedupe, `failed` counts per-user but the gate still compared to
pre-dedupe rules.length. 60 rules → 10 users → 2 failed writes =
20% real failure hidden behind a 60-rule denominator.
Fix: denominator is now eligibleUserCount (Map size after
group-and-filter). Log line reports rules + eligible_users +
success + skipped_empty + failed + duration so ops can see the
full shape.
Tests: 86/86 (was 83; +3 new: opt-out+sibling, all-opted-out drop,
candidate-ordering). Typecheck clean, node --check clean, biome clean.
* fix(brief): body-POST SETEX + attempted-only failure denominator
Addresses two sixth-round review findings on PR #3154.
1. Upstash SETEX (todo 228). The previous write path URL-encoded the
full envelope into /setex/{key}/{ttl}/{payload} which can blow
past proxy/edge/Node HTTP request-target limits for realistic
12-story briefs (5-20 KB JSON). Switched to body-POST via the
existing `redisPipeline` helper — same transport every other
write in the repo uses. Per-command error surface is preserved:
the wrapper throws on null pipeline response or on a {error}
entry in the result array.
2. Failure-rate denominator (todo 229). Earlier round switched
denominator from pre-dedupe rules.length to eligibleUserCount,
but the numerator only counts users that actually reached a
write attempt. skipped_empty users inflate eligibleUserCount
without being able to fail, so 4/4 failed writes against 100
eligible (96 skipped_empty) reads as 4% and silently passes.
Denominator is now `success + failed` (attempted writes only).
Extracted shouldExitNonZero({success, failed}) so the denominator
contract lives in a pure function with 7 test cases:
- 0 failures → no exit
- 100% failure on small volume → exits
- 1/20 at exact 5% threshold → exits (documented boundary)
- 1/50 below threshold → no exit
- 2/10 above Math.max(1) floor → exits
- 1/1 single isolated failure → exits
- 0 attempted (no signal) → no exit
Tests: 93/93 (was 86; +7 threshold cases). Typecheck + lint clean.
This commit is contained in:
49
Dockerfile.seed-brief-composer
Normal file
49
Dockerfile.seed-brief-composer
Normal file
@@ -0,0 +1,49 @@
|
||||
# =============================================================================
|
||||
# WorldMonitor Brief Composer (Phase 3a)
|
||||
# =============================================================================
|
||||
# Runs scripts/seed-brief-composer.mjs as a standalone Railway cron.
|
||||
# Reads news:insights:v1 from Upstash, queries Convex for enabled
|
||||
# alert rules, writes brief:{userId}:{issueDate} back to Upstash.
|
||||
#
|
||||
# Required env (set in Railway service vars):
|
||||
# UPSTASH_REDIS_REST_URL
|
||||
# UPSTASH_REDIS_REST_TOKEN
|
||||
# CONVEX_URL (or CONVEX_SITE_URL)
|
||||
# RELAY_SHARED_SECRET
|
||||
# Optional:
|
||||
# BRIEF_COMPOSER_ENABLED=0 (kill switch during incidents)
|
||||
#
|
||||
# Runtime characteristics: ~1 Upstash GET + N SETEX. N ≈ enabled PRO
|
||||
# users. No LLM calls (Phase 3b adds those); no outbound fan-out
|
||||
# (Phase 3c). CPU-cheap; memory bound by topStories payload size.
|
||||
# =============================================================================
|
||||
|
||||
FROM node:22-alpine
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
# Install scripts/ runtime dependencies (undici for Intl, etc.). The
|
||||
# composer itself uses only built-ins + the repo's own modules, but
|
||||
# tsx is needed to execute .mjs with explicit type-stripping when
|
||||
# shared/ imports traverse into server/_shared/brief-render.js
|
||||
# (which does not use TS).
|
||||
COPY scripts/package.json scripts/package-lock.json ./scripts/
|
||||
RUN npm ci --prefix scripts --omit=dev
|
||||
|
||||
# Composer script
|
||||
COPY scripts/seed-brief-composer.mjs ./scripts/seed-brief-composer.mjs
|
||||
|
||||
# Shared contract + renderer validator (assembly step calls
|
||||
# assertBriefEnvelope to refuse to write a malformed envelope).
|
||||
COPY shared/brief-envelope.js ./shared/brief-envelope.js
|
||||
COPY shared/brief-envelope.d.ts ./shared/brief-envelope.d.ts
|
||||
COPY shared/brief-filter.js ./shared/brief-filter.js
|
||||
COPY shared/brief-filter.d.ts ./shared/brief-filter.d.ts
|
||||
COPY server/_shared/brief-render.js ./server/_shared/brief-render.js
|
||||
COPY server/_shared/brief-render.d.ts ./server/_shared/brief-render.d.ts
|
||||
|
||||
# Upstash REST helper (reused from api/_upstash-json.js)
|
||||
COPY api/_upstash-json.js ./api/_upstash-json.js
|
||||
COPY api/_seed-envelope.js ./api/_seed-envelope.js
|
||||
|
||||
CMD ["node", "scripts/seed-brief-composer.mjs"]
|
||||
393
scripts/seed-brief-composer.mjs
Normal file
393
scripts/seed-brief-composer.mjs
Normal file
@@ -0,0 +1,393 @@
|
||||
#!/usr/bin/env node
|
||||
/**
|
||||
* WorldMonitor Brief composer — Railway cron.
|
||||
*
|
||||
* Phase 3a of docs/plans/2026-04-17-003-feat-worldmonitor-brief-
|
||||
* magazine-plan.md. Produces the per-user envelopes that Phases 1+2
|
||||
* already know how to serve; Phase 3b will replace the stubbed
|
||||
* digest text with LLM output.
|
||||
*
|
||||
* Per run:
|
||||
* 1. Fetch the global news-intelligence bundle once.
|
||||
* 2. Ask Convex for every enabled alert-rule with digestMode set.
|
||||
* This matches the eligibility set already used by
|
||||
* seed-digest-notifications — brief access is free-riding on
|
||||
* the digest opt-in.
|
||||
* 3. For each rule:
|
||||
* - Compute issueDate from rule.digestTimezone.
|
||||
* - Filter insights.topStories by rule.sensitivity.
|
||||
* - Assemble a BriefEnvelope with stubbed digest text.
|
||||
* - SETEX brief:{userId}:{issueDate} with a 7-day TTL.
|
||||
* 4. Log per-status counters (success / skipped_empty / failed).
|
||||
*
|
||||
* The script is idempotent within a day: re-running overwrites the
|
||||
* same key with the same envelope (modulo issuedAt). Phase 3c adds
|
||||
* fan-out events on first-write only.
|
||||
*/
|
||||
|
||||
import { createRequire } from 'node:module';
|
||||
import { fileURLToPath } from 'node:url';
|
||||
import { readRawJsonFromUpstash, redisPipeline } from '../api/_upstash-json.js';
|
||||
import {
|
||||
assembleStubbedBriefEnvelope,
|
||||
filterTopStories,
|
||||
issueDateInTz,
|
||||
} from '../shared/brief-filter.js';
|
||||
|
||||
const require = createRequire(import.meta.url);
|
||||
|
||||
// ── Config ────────────────────────────────────────────────────────────────────
|
||||
|
||||
const UPSTASH_URL = process.env.UPSTASH_REDIS_REST_URL ?? '';
|
||||
const UPSTASH_TOKEN = process.env.UPSTASH_REDIS_REST_TOKEN ?? '';
|
||||
const CONVEX_SITE_URL =
|
||||
process.env.CONVEX_SITE_URL ??
|
||||
(process.env.CONVEX_URL ?? '').replace('.convex.cloud', '.convex.site');
|
||||
const RELAY_SECRET = process.env.RELAY_SHARED_SECRET ?? '';
|
||||
|
||||
const BRIEF_TTL_SECONDS = 7 * 24 * 60 * 60; // 7 days
|
||||
const MAX_STORIES_PER_USER = 12;
|
||||
const INSIGHTS_KEY = 'news:insights:v1';
|
||||
|
||||
// ── Upstash helpers ──────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Write the brief envelope via the Upstash REST pipeline endpoint
|
||||
* (body-POST), not the path-embedded SETEX form. Realistic briefs
|
||||
* (12 stories, per-story description + whyMatters near caps) encode
|
||||
* to 5–20 KB of JSON; URL-encoding inflates that further and can hit
|
||||
* CDN / edge / Node HTTP request-target limits (commonly 8–16 KB).
|
||||
* `redisPipeline` places the command in a JSON body where size
|
||||
* limits are generous and uniform with the rest of the codebase's
|
||||
* Upstash writes.
|
||||
*/
|
||||
async function upstashSetex(key, value, ttlSeconds) {
|
||||
const results = await redisPipeline([
|
||||
['SETEX', key, String(ttlSeconds), JSON.stringify(value)],
|
||||
]);
|
||||
if (!results || !Array.isArray(results) || results.length === 0) {
|
||||
throw new Error(`Upstash SETEX failed for ${key}: null pipeline response`);
|
||||
}
|
||||
const result = results[0];
|
||||
// Upstash pipeline returns either {result} or {error} per command.
|
||||
if (result && typeof result === 'object' && 'error' in result) {
|
||||
throw new Error(`Upstash SETEX failed for ${key}: ${result.error}`);
|
||||
}
|
||||
return result;
|
||||
}
|
||||
|
||||
// ── Date helpers ─────────────────────────────────────────────────────────────
|
||||
|
||||
const MONTH_NAMES = [
|
||||
'January', 'February', 'March', 'April', 'May', 'June',
|
||||
'July', 'August', 'September', 'October', 'November', 'December',
|
||||
];
|
||||
|
||||
function dateLongFromIso(iso) {
|
||||
// iso is YYYY-MM-DD. Parse literally to avoid tz drift.
|
||||
const [y, m, d] = iso.split('-').map(Number);
|
||||
return `${d} ${MONTH_NAMES[m - 1]} ${y}`;
|
||||
}
|
||||
|
||||
function issueCodeFromIso(iso) {
|
||||
// "2026-04-18" → "18.04"
|
||||
const [, m, d] = iso.split('-');
|
||||
return `${d}.${m}`;
|
||||
}
|
||||
|
||||
function localHourInTz(nowMs, timezone) {
|
||||
try {
|
||||
const fmt = new Intl.DateTimeFormat('en-US', {
|
||||
timeZone: timezone,
|
||||
hour: 'numeric',
|
||||
hour12: false,
|
||||
});
|
||||
const hour = fmt.formatToParts(new Date(nowMs))
|
||||
.find((p) => p.type === 'hour')?.value;
|
||||
const n = Number(hour);
|
||||
return Number.isFinite(n) ? n : 9;
|
||||
} catch {
|
||||
return 9;
|
||||
}
|
||||
}
|
||||
|
||||
// ── Convex helpers ───────────────────────────────────────────────────────────
|
||||
|
||||
async function fetchDigestRules() {
|
||||
const res = await fetch(`${CONVEX_SITE_URL}/relay/digest-rules`, {
|
||||
method: 'GET',
|
||||
headers: {
|
||||
Authorization: `Bearer ${RELAY_SECRET}`,
|
||||
'User-Agent': 'worldmonitor-brief-composer/1.0',
|
||||
},
|
||||
signal: AbortSignal.timeout(10_000),
|
||||
});
|
||||
if (!res.ok) {
|
||||
throw new Error(`Failed to fetch digest rules: HTTP ${res.status}`);
|
||||
}
|
||||
const rules = await res.json();
|
||||
if (!Array.isArray(rules)) {
|
||||
throw new Error('digest-rules response was not an array');
|
||||
}
|
||||
return rules;
|
||||
}
|
||||
|
||||
// ── Failure gate ─────────────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* Decide whether the cron should exit non-zero so Railway flags the
|
||||
* run. Denominator is ATTEMPTED writes (success + failed); skipped-
|
||||
* empty users never reached the write path and must not inflate it.
|
||||
* Exported so the denominator contract is testable without mocking
|
||||
* Redis + LLM + the whole cron.
|
||||
*
|
||||
* @param {{ success: number; failed: number; thresholdRatio?: number }} counters
|
||||
* @returns {boolean}
|
||||
*/
|
||||
export function shouldExitNonZero({ success, failed, thresholdRatio = 0.05 }) {
|
||||
if (failed <= 0) return false;
|
||||
const attempted = success + failed;
|
||||
if (attempted <= 0) return false;
|
||||
const threshold = Math.max(1, Math.floor(attempted * thresholdRatio));
|
||||
return failed >= threshold;
|
||||
}
|
||||
|
||||
// ── User-name lookup (best effort) ───────────────────────────────────────────
|
||||
|
||||
function userDisplayNameFromId(userId) {
|
||||
// Clerk IDs look like "user_2abc..." — not display-friendly. Phase
|
||||
// 3b will hydrate names via a Convex query; Phase 3a uses a
|
||||
// generic "you" so the greeting still reads naturally without a
|
||||
// round-trip we don't yet need.
|
||||
void userId;
|
||||
return 'Reader';
|
||||
}
|
||||
|
||||
// ── Rule dedupe (one brief per user, not per variant) ───────────────────────
|
||||
|
||||
// Most-permissive-first ranking. Lower = broader.
|
||||
const SENSITIVITY_RANK = { all: 0, high: 1, critical: 2 };
|
||||
|
||||
function compareRules(a, b) {
|
||||
// Prefer the 'full' variant — it's the superset dashboard.
|
||||
const aFull = a.variant === 'full' ? 0 : 1;
|
||||
const bFull = b.variant === 'full' ? 0 : 1;
|
||||
if (aFull !== bFull) return aFull - bFull;
|
||||
// Tie-break on most permissive sensitivity (broadest brief).
|
||||
const aRank = SENSITIVITY_RANK[a.sensitivity ?? 'all'] ?? 0;
|
||||
const bRank = SENSITIVITY_RANK[b.sensitivity ?? 'all'] ?? 0;
|
||||
if (aRank !== bRank) return aRank - bRank;
|
||||
// Final tie-break: earlier-updated rule wins for determinism.
|
||||
return (a.updatedAt ?? 0) - (b.updatedAt ?? 0);
|
||||
}
|
||||
|
||||
/**
|
||||
* Group eligible (non-opted-out) rules by userId, with each user's
|
||||
* candidates sorted in preference order (best first). Returns an
|
||||
* array of `[userId, ranked-candidates[]]` pairs so the main loop
|
||||
* can try each variant in order and fall back when the preferred
|
||||
* one produces zero stories.
|
||||
*
|
||||
* aiDigestEnabled is pre-filtered here so a user whose preferred
|
||||
* variant is opted out but another variant is opted in still
|
||||
* produces a brief — the dedupe must not pick a variant that can
|
||||
* never emit.
|
||||
*/
|
||||
export function groupEligibleRulesByUser(rules) {
|
||||
/** @type {Map<string, any[]>} */
|
||||
const byUser = new Map();
|
||||
for (const rule of rules) {
|
||||
if (!rule || typeof rule.userId !== 'string') continue;
|
||||
// Default is OPT-IN — only an explicit false opts the user out.
|
||||
if (rule.aiDigestEnabled === false) continue;
|
||||
const list = byUser.get(rule.userId);
|
||||
if (list) list.push(rule);
|
||||
else byUser.set(rule.userId, [rule]);
|
||||
}
|
||||
for (const list of byUser.values()) {
|
||||
list.sort(compareRules);
|
||||
}
|
||||
return byUser;
|
||||
}
|
||||
|
||||
/**
|
||||
* @deprecated Kept so the existing dedupe tests still compile.
|
||||
* Prefer groupEligibleRulesByUser + per-user fallback in callers.
|
||||
*/
|
||||
export function dedupeRulesByUser(rules) {
|
||||
const grouped = groupEligibleRulesByUser(rules);
|
||||
const out = [];
|
||||
for (const candidates of grouped.values()) {
|
||||
if (candidates.length > 0) out.push(candidates[0]);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// ── Insights fetch ───────────────────────────────────────────────────────────
|
||||
|
||||
function extractInsights(raw) {
|
||||
// news:insights:v1 is stored as a seed envelope {_seed, data}.
|
||||
// readRawJsonFromUpstash intentionally does not unwrap; do so here.
|
||||
const data = raw?.data ?? raw;
|
||||
const topStories = Array.isArray(data?.topStories) ? data.topStories : [];
|
||||
const clusterCount = Number.isFinite(data?.clusterCount) ? data.clusterCount : topStories.length;
|
||||
const multiSourceCount = Number.isFinite(data?.multiSourceCount) ? data.multiSourceCount : 0;
|
||||
return {
|
||||
topStories,
|
||||
numbers: {
|
||||
clusters: clusterCount,
|
||||
multiSource: multiSourceCount,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
// ── SIGTERM handling ─────────────────────────────────────────────────────────
|
||||
// Matches the bundle-runner SIGTERM pattern (feedback note
|
||||
// bundle-runner-sigkill-leaks-child-lock). This script does not take
|
||||
// a distributed lock, but it does perform many parallel Upstash
|
||||
// writes; SIGTERM during the loop should flush partial progress
|
||||
// cleanly instead of throwing mid-fetch.
|
||||
let shuttingDown = false;
|
||||
process.on('SIGTERM', () => {
|
||||
shuttingDown = true;
|
||||
console.log('[brief-composer] SIGTERM received — finishing current iteration');
|
||||
});
|
||||
|
||||
// ── Main ─────────────────────────────────────────────────────────────────────
|
||||
|
||||
async function main() {
|
||||
const startMs = Date.now();
|
||||
console.log('[brief-composer] Run start:', new Date(startMs).toISOString());
|
||||
|
||||
let insightsRaw;
|
||||
try {
|
||||
insightsRaw = await readRawJsonFromUpstash(INSIGHTS_KEY);
|
||||
} catch (err) {
|
||||
console.error('[brief-composer] failed to read', INSIGHTS_KEY, err.message);
|
||||
process.exit(1);
|
||||
}
|
||||
if (!insightsRaw) {
|
||||
console.warn('[brief-composer] insights key empty; no brief to compose');
|
||||
return;
|
||||
}
|
||||
|
||||
const insights = extractInsights(insightsRaw);
|
||||
if (insights.topStories.length === 0) {
|
||||
console.warn('[brief-composer] upstream topStories empty; no brief to compose');
|
||||
return;
|
||||
}
|
||||
|
||||
let rules;
|
||||
try {
|
||||
rules = await fetchDigestRules();
|
||||
} catch (err) {
|
||||
console.error('[brief-composer]', err.message);
|
||||
process.exit(1);
|
||||
}
|
||||
console.log(`[brief-composer] Rules to process: ${rules.length}`);
|
||||
|
||||
// Briefs are user-scoped, but alertRules are (userId, variant)-scoped.
|
||||
// Group eligible (not-opted-out) rules by user in preference order
|
||||
// so we can fall back across variants when the preferred one can't
|
||||
// emit (opt-out on that variant, or zero matching stories).
|
||||
const eligibleByUser = groupEligibleRulesByUser(rules);
|
||||
|
||||
let success = 0;
|
||||
let skippedEmpty = 0;
|
||||
let failed = 0;
|
||||
|
||||
for (const [userId, candidates] of eligibleByUser) {
|
||||
if (shuttingDown) break;
|
||||
try {
|
||||
// Walk preference order; first variant with non-empty stories wins.
|
||||
let chosen = null;
|
||||
let chosenStories = null;
|
||||
for (const candidate of candidates) {
|
||||
const sensitivity = candidate.sensitivity ?? 'all';
|
||||
const stories = filterTopStories({
|
||||
stories: insights.topStories,
|
||||
sensitivity,
|
||||
maxStories: MAX_STORIES_PER_USER,
|
||||
});
|
||||
if (stories.length > 0) {
|
||||
chosen = candidate;
|
||||
chosenStories = stories;
|
||||
break;
|
||||
}
|
||||
}
|
||||
if (!chosen) {
|
||||
skippedEmpty += 1;
|
||||
continue;
|
||||
}
|
||||
if (candidates.length > 1) {
|
||||
console.log(
|
||||
`[brief-composer] dedup: userId=${userId} chose variant=${chosen.variant} sensitivity=${chosen.sensitivity ?? 'all'} from ${candidates.length} enabled variants`,
|
||||
);
|
||||
}
|
||||
|
||||
const tz = chosen.digestTimezone ?? 'UTC';
|
||||
const issueDate = issueDateInTz(startMs, tz);
|
||||
const envelope = assembleStubbedBriefEnvelope({
|
||||
user: { name: userDisplayNameFromId(chosen.userId), tz },
|
||||
stories: chosenStories,
|
||||
issueDate,
|
||||
dateLong: dateLongFromIso(issueDate),
|
||||
issue: issueCodeFromIso(issueDate),
|
||||
insightsNumbers: insights.numbers,
|
||||
issuedAt: Date.now(),
|
||||
localHour: localHourInTz(startMs, tz),
|
||||
});
|
||||
|
||||
const key = `brief:${chosen.userId}:${issueDate}`;
|
||||
await upstashSetex(key, envelope, BRIEF_TTL_SECONDS);
|
||||
success += 1;
|
||||
} catch (err) {
|
||||
failed += 1;
|
||||
const variants = candidates.map((c) => c.variant).join(',');
|
||||
console.error(
|
||||
`[brief-composer] failed for user=${userId} variants=${variants}:`,
|
||||
err.message,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
const eligibleUserCount = eligibleByUser.size;
|
||||
const attempted = success + failed;
|
||||
const durationMs = Date.now() - startMs;
|
||||
console.log(
|
||||
`[brief-composer] Done: rules=${rules.length} eligible_users=${eligibleUserCount} attempted=${attempted} success=${success} skipped_empty=${skippedEmpty} failed=${failed} duration_ms=${durationMs}`,
|
||||
);
|
||||
|
||||
if (shouldExitNonZero({ success, failed })) process.exit(1);
|
||||
}
|
||||
|
||||
// Only run the cron loop when executed as a script, never on import.
|
||||
// Tests import this file for the dedupe helpers and must not trigger
|
||||
// process.exit() at module load. Matches feedback_seed_isMain_guard.
|
||||
function isMain() {
|
||||
if (!process.argv[1]) return false;
|
||||
try {
|
||||
return fileURLToPath(import.meta.url) === process.argv[1];
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
if (isMain()) {
|
||||
if (process.env.BRIEF_COMPOSER_ENABLED === '0') {
|
||||
console.log('[brief-composer] BRIEF_COMPOSER_ENABLED=0 — skipping run');
|
||||
process.exit(0);
|
||||
}
|
||||
if (!UPSTASH_URL || !UPSTASH_TOKEN) {
|
||||
console.error('[brief-composer] UPSTASH_REDIS_REST_URL/TOKEN not set');
|
||||
process.exit(1);
|
||||
}
|
||||
if (!CONVEX_SITE_URL || !RELAY_SECRET) {
|
||||
console.error('[brief-composer] CONVEX_SITE_URL / RELAY_SHARED_SECRET not set');
|
||||
process.exit(1);
|
||||
}
|
||||
main().catch((err) => {
|
||||
console.error('[brief-composer] fatal:', err);
|
||||
process.exit(1);
|
||||
});
|
||||
}
|
||||
69
shared/brief-filter.d.ts
vendored
Normal file
69
shared/brief-filter.d.ts
vendored
Normal file
@@ -0,0 +1,69 @@
|
||||
// Pure helpers for composing a WorldMonitor Brief envelope from the
|
||||
// upstream news:insights:v1 cache + a user's alert-rule preferences.
|
||||
//
|
||||
// Split into its own module so Phase 3a (stubbed digest text) and
|
||||
// Phase 3b (LLM-generated digest) can share the same filter + shape
|
||||
// logic. Also importable from tests without pulling in Railway
|
||||
// runtime deps.
|
||||
|
||||
import type {
|
||||
BriefEnvelope,
|
||||
BriefStory,
|
||||
BriefThreatLevel,
|
||||
} from './brief-envelope.js';
|
||||
|
||||
/**
|
||||
* Upstream `news:insights:v1.topStories[i].threatLevel` uses an
|
||||
* extended ladder that includes 'moderate' as a synonym for
|
||||
* 'medium'. Phase 1 of the brief contract pinned the union to four
|
||||
* values; this helper normalises incoming severities.
|
||||
*/
|
||||
export function normaliseThreatLevel(upstream: string): BriefThreatLevel | null;
|
||||
|
||||
export type AlertSensitivity = 'all' | 'high' | 'critical';
|
||||
|
||||
/**
|
||||
* Filters the upstream `topStories` array against a user's
|
||||
* `alertRules.sensitivity` setting and caps at `maxStories`. Stories
|
||||
* with an unknown upstream severity are dropped.
|
||||
*/
|
||||
export function filterTopStories(input: {
|
||||
stories: UpstreamTopStory[];
|
||||
sensitivity: AlertSensitivity;
|
||||
maxStories?: number;
|
||||
}): BriefStory[];
|
||||
|
||||
/**
|
||||
* Builds a complete BriefEnvelope with stubbed digest text. Phase 3b
|
||||
* replaces the stubs with LLM output; every other field is final.
|
||||
*
|
||||
* Throws if the resulting envelope would fail assertBriefEnvelope —
|
||||
* the composer never writes an envelope the renderer cannot serve.
|
||||
*/
|
||||
export function assembleStubbedBriefEnvelope(input: {
|
||||
user: { name: string; tz: string };
|
||||
stories: BriefStory[];
|
||||
issueDate: string;
|
||||
dateLong: string;
|
||||
issue: string;
|
||||
insightsNumbers: { clusters: number; multiSource: number };
|
||||
issuedAt?: number;
|
||||
}): BriefEnvelope;
|
||||
|
||||
/**
|
||||
* Computes the user's local issue date from the current timestamp
|
||||
* and their IANA timezone. Falls back to UTC today for malformed
|
||||
* timezones so a composer run never blocks on one bad record.
|
||||
*/
|
||||
export function issueDateInTz(nowMs: number, timezone: string): string;
|
||||
|
||||
/** Upstream shape from news:insights:v1.topStories[]. */
|
||||
export interface UpstreamTopStory {
|
||||
primaryTitle?: unknown;
|
||||
primarySource?: unknown;
|
||||
description?: unknown;
|
||||
threatLevel?: unknown;
|
||||
category?: unknown;
|
||||
countryCode?: unknown;
|
||||
importanceScore?: unknown;
|
||||
}
|
||||
236
shared/brief-filter.js
Normal file
236
shared/brief-filter.js
Normal file
@@ -0,0 +1,236 @@
|
||||
// Pure helpers for composing a WorldMonitor Brief envelope from
|
||||
// upstream news:insights:v1 content + a user's alert-rule preferences.
|
||||
//
|
||||
// Split into its own module so Phase 3a (stubbed digest text) and
|
||||
// Phase 3b (LLM-generated digest) share the same filter + shape
|
||||
// logic. No I/O, no LLM calls, no network — fully testable.
|
||||
|
||||
import { BRIEF_ENVELOPE_VERSION } from './brief-envelope.js';
|
||||
import { assertBriefEnvelope } from '../server/_shared/brief-render.js';
|
||||
|
||||
/**
|
||||
* @typedef {import('./brief-envelope.js').BriefEnvelope} BriefEnvelope
|
||||
* @typedef {import('./brief-envelope.js').BriefStory} BriefStory
|
||||
* @typedef {import('./brief-envelope.js').BriefThreatLevel} BriefThreatLevel
|
||||
* @typedef {import('./brief-envelope.js').BriefThread} BriefThread
|
||||
* @typedef {import('./brief-envelope.js').BriefDigest} BriefDigest
|
||||
* @typedef {import('./brief-filter.js').AlertSensitivity} AlertSensitivity
|
||||
* @typedef {import('./brief-filter.js').UpstreamTopStory} UpstreamTopStory
|
||||
*/
|
||||
|
||||
// ── Severity normalisation ───────────────────────────────────────────────────
|
||||
|
||||
/** @type {Record<string, BriefThreatLevel>} */
|
||||
const SEVERITY_MAP = {
|
||||
critical: 'critical',
|
||||
high: 'high',
|
||||
medium: 'medium',
|
||||
// Upstream seed-insights still emits 'moderate' — alias to 'medium'.
|
||||
moderate: 'medium',
|
||||
low: 'low',
|
||||
};
|
||||
|
||||
/**
|
||||
* @param {unknown} upstream
|
||||
* @returns {BriefThreatLevel | null}
|
||||
*/
|
||||
export function normaliseThreatLevel(upstream) {
|
||||
if (typeof upstream !== 'string') return null;
|
||||
return SEVERITY_MAP[upstream.toLowerCase()] ?? null;
|
||||
}
|
||||
|
||||
// ── Sensitivity → severity threshold ─────────────────────────────────────────
|
||||
|
||||
/** @type {Record<AlertSensitivity, Set<BriefThreatLevel>>} */
|
||||
const ALLOWED_LEVELS_BY_SENSITIVITY = {
|
||||
// Matches convex/constants.ts sensitivityValidator: 'all'|'high'|'critical'.
|
||||
all: new Set(['critical', 'high', 'medium', 'low']),
|
||||
high: new Set(['critical', 'high']),
|
||||
critical: new Set(['critical']),
|
||||
};
|
||||
|
||||
// ── Filter ───────────────────────────────────────────────────────────────────
|
||||
|
||||
const MAX_HEADLINE_LEN = 200;
|
||||
const MAX_DESCRIPTION_LEN = 400;
|
||||
const MAX_SOURCE_LEN = 120;
|
||||
|
||||
/** @param {unknown} v */
|
||||
function asTrimmedString(v) {
|
||||
if (typeof v !== 'string') return '';
|
||||
return v.trim();
|
||||
}
|
||||
|
||||
/** @param {string} v @param {number} cap */
|
||||
function clip(v, cap) {
|
||||
if (v.length <= cap) return v;
|
||||
return `${v.slice(0, cap - 1).trimEnd()}\u2026`;
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{ stories: UpstreamTopStory[]; sensitivity: AlertSensitivity; maxStories?: number }} input
|
||||
* @returns {BriefStory[]}
|
||||
*/
|
||||
export function filterTopStories({ stories, sensitivity, maxStories = 12 }) {
|
||||
if (!Array.isArray(stories)) return [];
|
||||
const allowed = ALLOWED_LEVELS_BY_SENSITIVITY[sensitivity];
|
||||
if (!allowed) return [];
|
||||
|
||||
/** @type {BriefStory[]} */
|
||||
const out = [];
|
||||
for (const raw of stories) {
|
||||
if (out.length >= maxStories) break;
|
||||
if (!raw || typeof raw !== 'object') continue;
|
||||
const threatLevel = normaliseThreatLevel(raw.threatLevel);
|
||||
if (!threatLevel || !allowed.has(threatLevel)) continue;
|
||||
|
||||
const headline = clip(asTrimmedString(raw.primaryTitle), MAX_HEADLINE_LEN);
|
||||
if (!headline) continue;
|
||||
|
||||
const description = clip(
|
||||
asTrimmedString(raw.description) || headline,
|
||||
MAX_DESCRIPTION_LEN,
|
||||
);
|
||||
const source = clip(
|
||||
asTrimmedString(raw.primarySource) || 'Multiple wires',
|
||||
MAX_SOURCE_LEN,
|
||||
);
|
||||
const category = asTrimmedString(raw.category) || 'General';
|
||||
const country = asTrimmedString(raw.countryCode) || 'Global';
|
||||
|
||||
out.push({
|
||||
category,
|
||||
country,
|
||||
threatLevel,
|
||||
headline,
|
||||
description,
|
||||
source,
|
||||
// Stubbed at Phase 3a. Phase 3b replaces this with an LLM-
|
||||
// generated per-user rationale. The renderer requires a non-
|
||||
// empty string, so we emit a generic fallback rather than
|
||||
// leaving the field blank.
|
||||
whyMatters:
|
||||
'Story flagged by your sensitivity settings. Open for context.',
|
||||
});
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// ── Envelope assembly (stubbed digest text) ─────────────────────────────────
|
||||
|
||||
function deriveThreadsFromStories(stories) {
|
||||
const byCategory = new Map();
|
||||
for (const s of stories) {
|
||||
const n = byCategory.get(s.category) ?? 0;
|
||||
byCategory.set(s.category, n + 1);
|
||||
}
|
||||
const sorted = [...byCategory.entries()].sort((a, b) => b[1] - a[1]);
|
||||
return sorted.slice(0, 6).map(([tag, count]) => ({
|
||||
tag,
|
||||
teaser:
|
||||
count === 1
|
||||
? 'One thread on the desk today.'
|
||||
: `${count} threads on the desk today.`,
|
||||
}));
|
||||
}
|
||||
|
||||
function greetingForHour(localHour) {
|
||||
if (localHour < 5 || localHour >= 22) return 'Good evening.';
|
||||
if (localHour < 12) return 'Good morning.';
|
||||
if (localHour < 18) return 'Good afternoon.';
|
||||
return 'Good evening.';
|
||||
}
|
||||
|
||||
/**
|
||||
* @param {{
|
||||
* user: { name: string; tz: string };
|
||||
* stories: BriefStory[];
|
||||
* issueDate: string;
|
||||
* dateLong: string;
|
||||
* issue: string;
|
||||
* insightsNumbers: { clusters: number; multiSource: number };
|
||||
* issuedAt?: number;
|
||||
* localHour?: number;
|
||||
* }} input
|
||||
* @returns {BriefEnvelope}
|
||||
*/
|
||||
export function assembleStubbedBriefEnvelope({
|
||||
user,
|
||||
stories,
|
||||
issueDate,
|
||||
dateLong,
|
||||
issue,
|
||||
insightsNumbers,
|
||||
issuedAt = Date.now(),
|
||||
localHour,
|
||||
}) {
|
||||
const greeting = greetingForHour(
|
||||
typeof localHour === 'number' ? localHour : 9,
|
||||
);
|
||||
|
||||
/** @type {BriefDigest} */
|
||||
const digest = {
|
||||
greeting,
|
||||
// Phase 3b swaps this with an LLM-generated executive summary.
|
||||
// Phase 3a uses a neutral placeholder so the magazine still
|
||||
// renders end-to-end.
|
||||
lead: `Today's brief surfaces ${stories.length} ${
|
||||
stories.length === 1 ? 'thread' : 'threads'
|
||||
} flagged by your sensitivity settings. Open any page to read the full editorial.`,
|
||||
numbers: {
|
||||
clusters: insightsNumbers.clusters,
|
||||
multiSource: insightsNumbers.multiSource,
|
||||
surfaced: stories.length,
|
||||
},
|
||||
threads: deriveThreadsFromStories(stories),
|
||||
// Signals-to-watch is intentionally empty at Phase 3a. The
|
||||
// Digest / 04 Signals page is conditional in the renderer, so
|
||||
// an empty array simply drops that page instead of rendering
|
||||
// stubbed content that would read as noise.
|
||||
signals: [],
|
||||
};
|
||||
|
||||
/** @type {BriefEnvelope} */
|
||||
const envelope = {
|
||||
version: BRIEF_ENVELOPE_VERSION,
|
||||
issuedAt,
|
||||
data: {
|
||||
user,
|
||||
issue,
|
||||
date: issueDate,
|
||||
dateLong,
|
||||
digest,
|
||||
stories,
|
||||
},
|
||||
};
|
||||
|
||||
// Fail loud if the composer would produce an envelope the
|
||||
// renderer cannot serve. Phase 1 established this as the central
|
||||
// contract; drift here is the error mode we most care about.
|
||||
assertBriefEnvelope(envelope);
|
||||
return envelope;
|
||||
}
|
||||
|
||||
// ── Tz-aware issue date ──────────────────────────────────────────────────────
|
||||
|
||||
/**
|
||||
* @param {number} nowMs
|
||||
* @param {string} timezone
|
||||
* @returns {string}
|
||||
*/
|
||||
export function issueDateInTz(nowMs, timezone) {
|
||||
try {
|
||||
const fmt = new Intl.DateTimeFormat('en-CA', {
|
||||
timeZone: timezone,
|
||||
year: 'numeric',
|
||||
month: '2-digit',
|
||||
day: '2-digit',
|
||||
});
|
||||
// en-CA conveniently formats as YYYY-MM-DD.
|
||||
const parts = fmt.format(new Date(nowMs));
|
||||
if (/^\d{4}-\d{2}-\d{2}$/.test(parts)) return parts;
|
||||
} catch {
|
||||
/* fall through to UTC */
|
||||
}
|
||||
return new Date(nowMs).toISOString().slice(0, 10);
|
||||
}
|
||||
204
tests/brief-composer-rule-dedup.test.mjs
Normal file
204
tests/brief-composer-rule-dedup.test.mjs
Normal file
@@ -0,0 +1,204 @@
|
||||
// Regression tests for the Phase 3a composer's rule-selection logic.
|
||||
//
|
||||
// Two guards:
|
||||
// 1. aiDigestEnabled default parity — undefined must be opt-IN, matching
|
||||
// seed-digest-notifications.mjs:914 and notifications-settings.ts:228.
|
||||
// 2. Per-user dedupe — alertRules are (userId, variant)-scoped but the
|
||||
// brief key is user-scoped. Multi-variant users must produce exactly
|
||||
// one brief per issue, with a deterministic tie-breaker.
|
||||
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
dedupeRulesByUser,
|
||||
groupEligibleRulesByUser,
|
||||
shouldExitNonZero,
|
||||
} from '../scripts/seed-brief-composer.mjs';
|
||||
|
||||
function rule(overrides = {}) {
|
||||
return {
|
||||
userId: 'user_abc',
|
||||
variant: 'full',
|
||||
enabled: true,
|
||||
digestMode: 'daily',
|
||||
sensitivity: 'high',
|
||||
aiDigestEnabled: true,
|
||||
digestTimezone: 'UTC',
|
||||
updatedAt: 1_700_000_000_000,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe('dedupeRulesByUser', () => {
|
||||
it('keeps a single rule unchanged', () => {
|
||||
const out = dedupeRulesByUser([rule()]);
|
||||
assert.equal(out.length, 1);
|
||||
assert.equal(out[0].variant, 'full');
|
||||
});
|
||||
|
||||
it('dedupes multi-variant users to one rule, preferring "full"', () => {
|
||||
const out = dedupeRulesByUser([
|
||||
rule({ variant: 'finance', sensitivity: 'high' }),
|
||||
rule({ variant: 'full', sensitivity: 'critical' }),
|
||||
rule({ variant: 'tech', sensitivity: 'all' }),
|
||||
]);
|
||||
assert.equal(out.length, 1);
|
||||
assert.equal(out[0].variant, 'full');
|
||||
});
|
||||
|
||||
it('when no full variant: picks most permissive sensitivity', () => {
|
||||
const out = dedupeRulesByUser([
|
||||
rule({ variant: 'tech', sensitivity: 'critical' }),
|
||||
rule({ variant: 'finance', sensitivity: 'all' }),
|
||||
rule({ variant: 'energy', sensitivity: 'high' }),
|
||||
]);
|
||||
assert.equal(out.length, 1);
|
||||
// 'all' is the most permissive.
|
||||
assert.equal(out[0].variant, 'finance');
|
||||
});
|
||||
|
||||
it('never cross-contaminates across userIds', () => {
|
||||
const out = dedupeRulesByUser([
|
||||
rule({ userId: 'user_a', variant: 'full' }),
|
||||
rule({ userId: 'user_b', variant: 'tech' }),
|
||||
rule({ userId: 'user_a', variant: 'finance' }),
|
||||
]);
|
||||
assert.equal(out.length, 2);
|
||||
const a = out.find((r) => r.userId === 'user_a');
|
||||
const b = out.find((r) => r.userId === 'user_b');
|
||||
assert.equal(a.variant, 'full');
|
||||
assert.equal(b.variant, 'tech');
|
||||
});
|
||||
|
||||
it('drops rules without a string userId', () => {
|
||||
const out = dedupeRulesByUser([
|
||||
rule({ userId: /** @type {any} */ (null) }),
|
||||
rule({ userId: 'user_ok' }),
|
||||
]);
|
||||
assert.equal(out.length, 1);
|
||||
assert.equal(out[0].userId, 'user_ok');
|
||||
});
|
||||
|
||||
it('is deterministic across duplicate full-variant rules via updatedAt tie-breaker', () => {
|
||||
const older = rule({ variant: 'full', sensitivity: 'high', updatedAt: 1_000 });
|
||||
const newer = rule({ variant: 'full', sensitivity: 'high', updatedAt: 2_000 });
|
||||
const out1 = dedupeRulesByUser([older, newer]);
|
||||
const out2 = dedupeRulesByUser([newer, older]);
|
||||
// Earlier updatedAt wins — stable under input reordering.
|
||||
assert.equal(out1[0].updatedAt, 1_000);
|
||||
assert.equal(out2[0].updatedAt, 1_000);
|
||||
});
|
||||
});
|
||||
|
||||
describe('aiDigestEnabled default parity', () => {
|
||||
// The composer's main loop short-circuits on `rule.aiDigestEnabled
|
||||
// === false`. Exercising the predicate directly so a refactor that
|
||||
// re-inverts it (back to `!rule.aiDigestEnabled`) fails loud.
|
||||
|
||||
function shouldSkipForAiDigest(rule) {
|
||||
return rule.aiDigestEnabled === false;
|
||||
}
|
||||
|
||||
it('includes rules with aiDigestEnabled: true', () => {
|
||||
assert.equal(shouldSkipForAiDigest({ aiDigestEnabled: true }), false);
|
||||
});
|
||||
|
||||
it('includes rules with aiDigestEnabled: undefined (legacy rows)', () => {
|
||||
assert.equal(shouldSkipForAiDigest({ aiDigestEnabled: undefined }), false);
|
||||
});
|
||||
|
||||
it('includes rules with no aiDigestEnabled field at all (legacy rows)', () => {
|
||||
assert.equal(shouldSkipForAiDigest({}), false);
|
||||
});
|
||||
|
||||
it('excludes only when explicitly false', () => {
|
||||
assert.equal(shouldSkipForAiDigest({ aiDigestEnabled: false }), true);
|
||||
});
|
||||
|
||||
it('groupEligibleRulesByUser: opted-out preferred variant falls back to opted-in sibling', () => {
|
||||
const grouped = groupEligibleRulesByUser([
|
||||
rule({ variant: 'full', aiDigestEnabled: false, updatedAt: 100 }),
|
||||
rule({ variant: 'finance', aiDigestEnabled: true, updatedAt: 200 }),
|
||||
]);
|
||||
const candidates = grouped.get('user_abc');
|
||||
assert.ok(candidates, 'user is still eligible via the opt-in variant');
|
||||
assert.equal(candidates.length, 1);
|
||||
assert.equal(candidates[0].variant, 'finance');
|
||||
});
|
||||
|
||||
it('groupEligibleRulesByUser: user with all variants opted-out is dropped entirely', () => {
|
||||
const grouped = groupEligibleRulesByUser([
|
||||
rule({ variant: 'full', aiDigestEnabled: false }),
|
||||
rule({ variant: 'finance', aiDigestEnabled: false }),
|
||||
]);
|
||||
assert.equal(grouped.size, 0);
|
||||
});
|
||||
|
||||
it('groupEligibleRulesByUser: retains all eligible candidates in preference order', () => {
|
||||
const grouped = groupEligibleRulesByUser([
|
||||
rule({ variant: 'finance', sensitivity: 'critical', updatedAt: 100 }),
|
||||
rule({ variant: 'full', sensitivity: 'critical', updatedAt: 200 }),
|
||||
rule({ variant: 'tech', sensitivity: 'all', updatedAt: 300 }),
|
||||
]);
|
||||
const candidates = grouped.get('user_abc');
|
||||
assert.equal(candidates.length, 3);
|
||||
// First is full (preferred variant); then tech (most permissive sensitivity);
|
||||
// then finance. Fallback loop in the main() script tries them in this order.
|
||||
assert.equal(candidates[0].variant, 'full');
|
||||
assert.equal(candidates[1].variant, 'tech');
|
||||
assert.equal(candidates[2].variant, 'finance');
|
||||
});
|
||||
|
||||
it('shouldExitNonZero: returns false when no failures', () => {
|
||||
assert.equal(shouldExitNonZero({ success: 10, failed: 0 }), false);
|
||||
});
|
||||
|
||||
it('shouldExitNonZero: catches 100% failure on small attempted volume', () => {
|
||||
// 4 attempted, 4 failed, 96 eligible skipped-empty. The earlier
|
||||
// (eligibleUserCount) denominator would read 4/100=4% and pass.
|
||||
assert.equal(shouldExitNonZero({ success: 0, failed: 4 }), true);
|
||||
});
|
||||
|
||||
it('shouldExitNonZero: 1/20 failures is exactly at 5% (floor(20*0.05)=1), trips', () => {
|
||||
// Exact-threshold boundary: documents intentional behaviour.
|
||||
assert.equal(shouldExitNonZero({ success: 19, failed: 1 }), true);
|
||||
});
|
||||
|
||||
it('shouldExitNonZero: 1/50 failures stays under threshold (floor(50*0.05)=2)', () => {
|
||||
// Threshold floor is Math.max(1, floor(N*0.05)). For N<40 a
|
||||
// single failure always trips. At N=50 the threshold is 2, so
|
||||
// 1/50 stays green. Ops intuition: the 5% bar is only a "bar"
|
||||
// once you have a meaningful sample.
|
||||
assert.equal(shouldExitNonZero({ success: 49, failed: 1 }), false);
|
||||
});
|
||||
|
||||
it('shouldExitNonZero: 2/10 exceeds threshold', () => {
|
||||
// floor(10 * 0.05) = 0 → Math.max forces 1. failed=2 >= 1.
|
||||
assert.equal(shouldExitNonZero({ success: 8, failed: 2 }), true);
|
||||
});
|
||||
|
||||
it('shouldExitNonZero: single isolated failure still tripwires', () => {
|
||||
// floor(1 * 0.05) = 0 → Math.max forces 1. failed=1 >= 1.
|
||||
assert.equal(shouldExitNonZero({ success: 0, failed: 1 }), true);
|
||||
});
|
||||
|
||||
it('shouldExitNonZero: zero attempted means no signal, returns false', () => {
|
||||
assert.equal(shouldExitNonZero({ success: 0, failed: 0 }), false);
|
||||
});
|
||||
|
||||
it('matches seed-digest-notifications convention', async () => {
|
||||
// Cross-reference: the existing digest cron uses the same
|
||||
// `!== false` test. If it drifts, the brief and digest will
|
||||
// disagree on who is eligible. This assertion lives here to
|
||||
// surface the divergence loudly.
|
||||
const fs = await import('node:fs/promises');
|
||||
const src = await fs.readFile(
|
||||
new URL('../scripts/seed-digest-notifications.mjs', import.meta.url),
|
||||
'utf8',
|
||||
);
|
||||
assert.ok(
|
||||
src.includes('rule.aiDigestEnabled !== false'),
|
||||
'seed-digest-notifications.mjs must keep `rule.aiDigestEnabled !== false`',
|
||||
);
|
||||
});
|
||||
});
|
||||
227
tests/brief-filter.test.mjs
Normal file
227
tests/brief-filter.test.mjs
Normal file
@@ -0,0 +1,227 @@
|
||||
// Pure-function tests for the Phase 3a brief composer helpers.
|
||||
//
|
||||
// Locks in: severity normalisation (moderate → medium), sensitivity
|
||||
// threshold, story cap, envelope assembly passes the renderer's
|
||||
// strict validator, threads derivation, tz-aware issue date.
|
||||
|
||||
import { describe, it } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import {
|
||||
normaliseThreatLevel,
|
||||
filterTopStories,
|
||||
assembleStubbedBriefEnvelope,
|
||||
issueDateInTz,
|
||||
} from '../shared/brief-filter.js';
|
||||
import { BRIEF_ENVELOPE_VERSION } from '../shared/brief-envelope.js';
|
||||
|
||||
function upstreamStory(overrides = {}) {
|
||||
return {
|
||||
primaryTitle: 'Iran declares Strait of Hormuz open. Oil drops more than 9%.',
|
||||
primarySource: 'Reuters',
|
||||
description: 'Tehran publicly reopened the Strait of Hormuz to commercial shipping today.',
|
||||
threatLevel: 'high',
|
||||
category: 'Energy',
|
||||
countryCode: 'IR',
|
||||
importanceScore: 320,
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
describe('normaliseThreatLevel', () => {
|
||||
it('accepts the four canonical values', () => {
|
||||
for (const level of ['critical', 'high', 'medium', 'low']) {
|
||||
assert.equal(normaliseThreatLevel(level), level);
|
||||
}
|
||||
});
|
||||
|
||||
it('maps upstream "moderate" to "medium"', () => {
|
||||
assert.equal(normaliseThreatLevel('moderate'), 'medium');
|
||||
});
|
||||
|
||||
it('case-insensitive', () => {
|
||||
assert.equal(normaliseThreatLevel('HIGH'), 'high');
|
||||
assert.equal(normaliseThreatLevel('Moderate'), 'medium');
|
||||
});
|
||||
|
||||
it('returns null on unknown or non-string input', () => {
|
||||
assert.equal(normaliseThreatLevel('unknown'), null);
|
||||
assert.equal(normaliseThreatLevel(null), null);
|
||||
assert.equal(normaliseThreatLevel(42), null);
|
||||
});
|
||||
});
|
||||
|
||||
describe('filterTopStories', () => {
|
||||
it('respects sensitivity=critical (keeps critical only)', () => {
|
||||
const out = filterTopStories({
|
||||
stories: [
|
||||
upstreamStory({ threatLevel: 'critical' }),
|
||||
upstreamStory({ threatLevel: 'high' }),
|
||||
upstreamStory({ threatLevel: 'medium' }),
|
||||
],
|
||||
sensitivity: 'critical',
|
||||
});
|
||||
assert.equal(out.length, 1);
|
||||
assert.equal(out[0].threatLevel, 'critical');
|
||||
});
|
||||
|
||||
it('sensitivity=high keeps critical + high', () => {
|
||||
const out = filterTopStories({
|
||||
stories: [
|
||||
upstreamStory({ threatLevel: 'critical' }),
|
||||
upstreamStory({ threatLevel: 'high' }),
|
||||
upstreamStory({ threatLevel: 'medium' }),
|
||||
upstreamStory({ threatLevel: 'low' }),
|
||||
],
|
||||
sensitivity: 'high',
|
||||
});
|
||||
assert.equal(out.length, 2);
|
||||
});
|
||||
|
||||
it('sensitivity=all keeps everything with a known severity', () => {
|
||||
const out = filterTopStories({
|
||||
stories: [
|
||||
upstreamStory({ threatLevel: 'critical' }),
|
||||
upstreamStory({ threatLevel: 'high' }),
|
||||
upstreamStory({ threatLevel: 'moderate' }),
|
||||
upstreamStory({ threatLevel: 'low' }),
|
||||
upstreamStory({ threatLevel: 'unknown' }),
|
||||
],
|
||||
sensitivity: 'all',
|
||||
});
|
||||
assert.equal(out.length, 4);
|
||||
});
|
||||
|
||||
it('caps at maxStories', () => {
|
||||
const stories = Array.from({ length: 20 }, (_, i) =>
|
||||
upstreamStory({ primaryTitle: `Story ${i}` }),
|
||||
);
|
||||
const out = filterTopStories({ stories, sensitivity: 'all', maxStories: 5 });
|
||||
assert.equal(out.length, 5);
|
||||
});
|
||||
|
||||
it('falls back to Multiple wires when primarySource missing', () => {
|
||||
const out = filterTopStories({
|
||||
stories: [upstreamStory({ primarySource: '' })],
|
||||
sensitivity: 'all',
|
||||
});
|
||||
assert.equal(out[0].source, 'Multiple wires');
|
||||
});
|
||||
|
||||
it('drops stories with empty primaryTitle', () => {
|
||||
const out = filterTopStories({
|
||||
stories: [upstreamStory({ primaryTitle: ' ' })],
|
||||
sensitivity: 'all',
|
||||
});
|
||||
assert.equal(out.length, 0);
|
||||
});
|
||||
|
||||
it('returns empty for unknown sensitivity', () => {
|
||||
const out = filterTopStories({
|
||||
stories: [upstreamStory()],
|
||||
sensitivity: /** @type {any} */ ('bogus'),
|
||||
});
|
||||
assert.equal(out.length, 0);
|
||||
});
|
||||
|
||||
it('non-array input returns empty', () => {
|
||||
assert.deepEqual(
|
||||
filterTopStories({
|
||||
stories: /** @type {any} */ (null),
|
||||
sensitivity: 'all',
|
||||
}),
|
||||
[],
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('assembleStubbedBriefEnvelope', () => {
|
||||
const baseStories = [
|
||||
upstreamStory({ threatLevel: 'critical' }),
|
||||
upstreamStory({ threatLevel: 'high', category: 'Diplomacy' }),
|
||||
upstreamStory({ threatLevel: 'high', category: 'Maritime' }),
|
||||
upstreamStory({ threatLevel: 'medium', category: 'Energy' }),
|
||||
];
|
||||
|
||||
function baseInput() {
|
||||
const stories = filterTopStories({
|
||||
stories: baseStories,
|
||||
sensitivity: 'all',
|
||||
});
|
||||
return {
|
||||
user: { name: 'Elie', tz: 'UTC' },
|
||||
stories,
|
||||
issueDate: '2026-04-18',
|
||||
dateLong: '18 April 2026',
|
||||
issue: '18.04',
|
||||
insightsNumbers: { clusters: 278, multiSource: 21 },
|
||||
issuedAt: 1_700_000_000_000,
|
||||
localHour: 9,
|
||||
};
|
||||
}
|
||||
|
||||
it('produces an envelope that passes the strict renderer validator', () => {
|
||||
const env = assembleStubbedBriefEnvelope(baseInput());
|
||||
assert.equal(env.version, BRIEF_ENVELOPE_VERSION);
|
||||
assert.equal(env.data.digest.numbers.surfaced, env.data.stories.length);
|
||||
assert.equal(env.data.digest.signals.length, 0);
|
||||
assert.ok(env.data.digest.threads.length > 0);
|
||||
});
|
||||
|
||||
it('morning greeting at hour 9', () => {
|
||||
const env = assembleStubbedBriefEnvelope({ ...baseInput(), localHour: 9 });
|
||||
assert.equal(env.data.digest.greeting, 'Good morning.');
|
||||
});
|
||||
|
||||
it('evening greeting at hour 22', () => {
|
||||
const env = assembleStubbedBriefEnvelope({ ...baseInput(), localHour: 22 });
|
||||
assert.equal(env.data.digest.greeting, 'Good evening.');
|
||||
});
|
||||
|
||||
it('afternoon greeting at hour 14', () => {
|
||||
const env = assembleStubbedBriefEnvelope({ ...baseInput(), localHour: 14 });
|
||||
assert.equal(env.data.digest.greeting, 'Good afternoon.');
|
||||
});
|
||||
|
||||
it('threads are derived from category frequency, capped at 6', () => {
|
||||
const many = Array.from({ length: 10 }, (_, i) =>
|
||||
upstreamStory({ category: `Cat${i}`, threatLevel: 'high' }),
|
||||
);
|
||||
const stories = filterTopStories({ stories: many, sensitivity: 'all' });
|
||||
const env = assembleStubbedBriefEnvelope({
|
||||
...baseInput(),
|
||||
stories,
|
||||
});
|
||||
assert.ok(env.data.digest.threads.length <= 6);
|
||||
});
|
||||
|
||||
it('throws when assembled envelope would fail validation (empty stories)', () => {
|
||||
assert.throws(() =>
|
||||
assembleStubbedBriefEnvelope({
|
||||
...baseInput(),
|
||||
stories: [],
|
||||
}),
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
describe('issueDateInTz', () => {
|
||||
// 2026-04-18T00:30:00Z — midnight UTC + 30min. Tokyo (+9) is
|
||||
// already mid-morning on the 18th; LA (-7) is late on the 17th.
|
||||
const midnightUtc = Date.UTC(2026, 3, 18, 0, 30, 0);
|
||||
|
||||
it('UTC returns the UTC date', () => {
|
||||
assert.equal(issueDateInTz(midnightUtc, 'UTC'), '2026-04-18');
|
||||
});
|
||||
|
||||
it('positive offset (Asia/Tokyo) returns the later local date', () => {
|
||||
assert.equal(issueDateInTz(midnightUtc, 'Asia/Tokyo'), '2026-04-18');
|
||||
});
|
||||
|
||||
it('negative offset (America/Los_Angeles) returns the earlier local date', () => {
|
||||
assert.equal(issueDateInTz(midnightUtc, 'America/Los_Angeles'), '2026-04-17');
|
||||
});
|
||||
|
||||
it('malformed timezone falls back to UTC', () => {
|
||||
assert.equal(issueDateInTz(midnightUtc, 'Not/A_Zone'), '2026-04-18');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user