feat(brief): per-user composer writing brief:{userId}:{issueDate} (Phase 3a) (#3154)

* feat(brief): per-user composer writing brief:{userId}:{issueDate} (Phase 3a)

Phase 3a of docs/plans/2026-04-17-003. Produces the Redis-resident
envelopes that Phases 1 (renderer) and 2 (edge routes) already know
how to serve, so after this ships the end-to-end read path works
with real data.

Files:

- shared/brief-filter.{js,d.ts}: pure helpers. normaliseThreatLevel
  maps upstream 'moderate' -> 'medium' (contract pinned the union in
  Phase 1). filterTopStories applies sensitivity thresholds and caps
  at maxStories. assembleStubbedBriefEnvelope builds a full envelope
  with stubbed greeting/lead/threads/signals and runs it through the
  renderer's assertBriefEnvelope so no malformed envelope is ever
  persisted. issueDateInTz computes per-user local date via Intl
  with UTC fallback.

- scripts/seed-brief-composer.mjs: Railway cron. Reads
  news:insights:v1 once, fetches enabled alert rules via the
  existing /relay/digest-rules endpoint (same set
  seed-digest-notifications uses), then for each rule computes the
  user's local issue date, filters stories, assembles an envelope,
  and SETEX brief:{userId}:{issueDate} with 7-day TTL. Respects
  aiDigestEnabled opt-in. Honours SIGTERM. Exits non-zero when >5%
  of rules fail so Railway surfaces structural breakage.

- Dockerfile.seed-brief-composer: standalone container. Copies the
  minimum set (composer + shared/ contract + renderer validator +
  Upstash helper + seed-envelope unwrapper).

- tests/brief-filter.test.mjs: 22 pure-function tests covering
  severity normalisation (including 'moderate' alias), sensitivity
  thresholds, story cap, empty-title drop, envelope assembly passes
  the strict renderer validator, tz-aware date math across +UTC/-UTC
  offsets with a bad-timezone fallback.

Out of scope for this PR:
- LLM-generated whyMatters / lead / signals (Phase 3b).
- brief_ready event fan-out to notification-relay (Phase 3c).
- Dashboard panel that consumes /api/latest-brief (Phase 4).

Pre-merge runbook:
1. Create a new Railway service from Dockerfile.seed-brief-composer.
2. Set env vars (UPSTASH_*, CONVEX_URL, RELAY_SHARED_SECRET) — reuse
   the values already in the digest service.
3. Add a cron schedule (suggested: hourly at :05 so it lands between
   the insights-seeder tick and the digest cron).
4. Verify first run: check service logs for
   "[brief-composer] Done: success=X ..." and a reader's
   /api/latest-brief should stop returning 'composing' within one
   cron cycle.

Tests: 72/72 (22 brief-filter + 30 render + 20 HMAC). Typecheck +
lint clean. Composer script parses with node --check.

* fix(brief): aiDigestEnabled default + per-user rule dedupe

Addresses two fourth-round review findings on PR #3154.

1. aiDigestEnabled default parity (todo 224). Composer was checking
   `!rule.aiDigestEnabled`, which skips legacy rules that predate the
   optional field. The rest of the codebase defaults it to true
   (seed-digest-notifications.mjs:914 uses `!== false`;
   notifications-settings.ts:228 uses `?? true`; the Convex setter
   defaults to true). Flipped the composer to `=== false` so only an
   explicit opt-out skips the brief.

2. Multi-variant last-write-wins (todo 225). alertRules are
   (userId, variant)-scoped but the brief key is user-scoped
   (brief:{userId}:{issueDate}). Users with the full+finance+tech
   variants all enabled would produce three competing writes with a
   nondeterministic survivor. Added dedupeRulesByUser() that picks
   one rule per user: prefers 'full' variant, then most permissive
   sensitivity (all > high > critical), tie-breaking on earliest
   updatedAt for stability across input reordering. Logs the
   occurrence so we can see how often users have multi-variant
   configs.

Also hardened against future regressions:

- Moved env-var guards + main() call behind an isMain() check
  (feedback_seed_isMain_guard). Previously, importing the script
  from a test would fire process.exit(0) on the
  BRIEF_COMPOSER_ENABLED=0 branch and kill the test runner. Tests
  now load the file cleanly.

- Exported dedupeRulesByUser so the tests can exercise the selection
  logic directly.

- The new tests/brief-composer-rule-dedup.test.mjs includes a
  cross-module assertion that seed-digest-notifications.mjs still
  reads `rule.aiDigestEnabled !== false`. If the digest cron ever
  drifts, this test fails loud — the brief and digest must agree on
  who is eligible.

Tests: 83/83 (was 72; +6 dedupe cases + 5 aiDigestEnabled parity
cases). Typecheck + lint clean.

* fix(brief): dedupe order + failure-rate denominator

Addresses two fifth-round review findings on PR #3154.

1. Dedupe was picking a preferred variant BEFORE checking whether it
   could actually emit a brief (todo 226). A user with
   aiDigestEnabled=false on 'full' but true on 'finance' got skipped
   entirely; same for a user with sensitivity='critical' on 'full'
   that filters to zero stories while 'finance' has matching content.

   Replaced dedupeRulesByUser with groupEligibleRulesByUser: pre-
   filters opted-out rules, then returns ALL eligible variants per
   user in preference order. The main loop walks candidates and
   takes the first one whose story filter produces non-empty content.
   Fallback is cheap (story filter is pure) and preserves the 'full'-
   first + most-permissive-sensitivity tie-breakers from before.

   dedupeRulesByUser is kept as a thin wrapper for the existing tests;
   new tests exercise the group+fallback path directly (opt-out +
   opt-in sibling, all-opted-out drop, ordering stability).

2. Failure gate denominator drifted from numerator (todo 227). After
   dedupe, `failed` counts per-user but the gate still compared to
   pre-dedupe rules.length. 60 rules → 10 users → 2 failed writes =
   20% real failure hidden behind a 60-rule denominator.

   Fix: denominator is now eligibleUserCount (Map size after
   group-and-filter). Log line reports rules + eligible_users +
   success + skipped_empty + failed + duration so ops can see the
   full shape.

Tests: 86/86 (was 83; +3 new: opt-out+sibling, all-opted-out drop,
candidate-ordering). Typecheck clean, node --check clean, biome clean.

* fix(brief): body-POST SETEX + attempted-only failure denominator

Addresses two sixth-round review findings on PR #3154.

1. Upstash SETEX (todo 228). The previous write path URL-encoded the
   full envelope into /setex/{key}/{ttl}/{payload} which can blow
   past proxy/edge/Node HTTP request-target limits for realistic
   12-story briefs (5-20 KB JSON). Switched to body-POST via the
   existing `redisPipeline` helper — same transport every other
   write in the repo uses. Per-command error surface is preserved:
   the wrapper throws on null pipeline response or on a {error}
   entry in the result array.

2. Failure-rate denominator (todo 229). Earlier round switched
   denominator from pre-dedupe rules.length to eligibleUserCount,
   but the numerator only counts users that actually reached a
   write attempt. skipped_empty users inflate eligibleUserCount
   without being able to fail, so 4/4 failed writes against 100
   eligible (96 skipped_empty) reads as 4% and silently passes.
   Denominator is now `success + failed` (attempted writes only).

Extracted shouldExitNonZero({success, failed}) so the denominator
contract lives in a pure function with 7 test cases:
- 0 failures → no exit
- 100% failure on small volume → exits
- 1/20 at exact 5% threshold → exits (documented boundary)
- 1/50 below threshold → no exit
- 2/10 above Math.max(1) floor → exits
- 1/1 single isolated failure → exits
- 0 attempted (no signal) → no exit

Tests: 93/93 (was 86; +7 threshold cases). Typecheck + lint clean.
This commit is contained in:
Elie Habib
2026-04-18 08:45:02 +04:00
committed by GitHub
parent de769ce8e1
commit 45da551d17
6 changed files with 1178 additions and 0 deletions

View File

@@ -0,0 +1,49 @@
# =============================================================================
# WorldMonitor Brief Composer (Phase 3a)
# =============================================================================
# Runs scripts/seed-brief-composer.mjs as a standalone Railway cron.
# Reads news:insights:v1 from Upstash, queries Convex for enabled
# alert rules, writes brief:{userId}:{issueDate} back to Upstash.
#
# Required env (set in Railway service vars):
# UPSTASH_REDIS_REST_URL
# UPSTASH_REDIS_REST_TOKEN
# CONVEX_URL (or CONVEX_SITE_URL)
# RELAY_SHARED_SECRET
# Optional:
# BRIEF_COMPOSER_ENABLED=0 (kill switch during incidents)
#
# Runtime characteristics: ~1 Upstash GET + N SETEX. N ≈ enabled PRO
# users. No LLM calls (Phase 3b adds those); no outbound fan-out
# (Phase 3c). CPU-cheap; memory bound by topStories payload size.
# =============================================================================
FROM node:22-alpine
WORKDIR /app
# Install scripts/ runtime dependencies (undici for Intl, etc.). The
# composer itself uses only built-ins + the repo's own modules, but
# tsx is needed to execute .mjs with explicit type-stripping when
# shared/ imports traverse into server/_shared/brief-render.js
# (which does not use TS).
COPY scripts/package.json scripts/package-lock.json ./scripts/
RUN npm ci --prefix scripts --omit=dev
# Composer script
COPY scripts/seed-brief-composer.mjs ./scripts/seed-brief-composer.mjs
# Shared contract + renderer validator (assembly step calls
# assertBriefEnvelope to refuse to write a malformed envelope).
COPY shared/brief-envelope.js ./shared/brief-envelope.js
COPY shared/brief-envelope.d.ts ./shared/brief-envelope.d.ts
COPY shared/brief-filter.js ./shared/brief-filter.js
COPY shared/brief-filter.d.ts ./shared/brief-filter.d.ts
COPY server/_shared/brief-render.js ./server/_shared/brief-render.js
COPY server/_shared/brief-render.d.ts ./server/_shared/brief-render.d.ts
# Upstash REST helper (reused from api/_upstash-json.js)
COPY api/_upstash-json.js ./api/_upstash-json.js
COPY api/_seed-envelope.js ./api/_seed-envelope.js
CMD ["node", "scripts/seed-brief-composer.mjs"]

View File

@@ -0,0 +1,393 @@
#!/usr/bin/env node
/**
* WorldMonitor Brief composer — Railway cron.
*
* Phase 3a of docs/plans/2026-04-17-003-feat-worldmonitor-brief-
* magazine-plan.md. Produces the per-user envelopes that Phases 1+2
* already know how to serve; Phase 3b will replace the stubbed
* digest text with LLM output.
*
* Per run:
* 1. Fetch the global news-intelligence bundle once.
* 2. Ask Convex for every enabled alert-rule with digestMode set.
* This matches the eligibility set already used by
* seed-digest-notifications — brief access is free-riding on
* the digest opt-in.
* 3. For each rule:
* - Compute issueDate from rule.digestTimezone.
* - Filter insights.topStories by rule.sensitivity.
* - Assemble a BriefEnvelope with stubbed digest text.
* - SETEX brief:{userId}:{issueDate} with a 7-day TTL.
* 4. Log per-status counters (success / skipped_empty / failed).
*
* The script is idempotent within a day: re-running overwrites the
* same key with the same envelope (modulo issuedAt). Phase 3c adds
* fan-out events on first-write only.
*/
import { createRequire } from 'node:module';
import { fileURLToPath } from 'node:url';
import { readRawJsonFromUpstash, redisPipeline } from '../api/_upstash-json.js';
import {
assembleStubbedBriefEnvelope,
filterTopStories,
issueDateInTz,
} from '../shared/brief-filter.js';
const require = createRequire(import.meta.url);
// ── Config ────────────────────────────────────────────────────────────────────
const UPSTASH_URL = process.env.UPSTASH_REDIS_REST_URL ?? '';
const UPSTASH_TOKEN = process.env.UPSTASH_REDIS_REST_TOKEN ?? '';
const CONVEX_SITE_URL =
process.env.CONVEX_SITE_URL ??
(process.env.CONVEX_URL ?? '').replace('.convex.cloud', '.convex.site');
const RELAY_SECRET = process.env.RELAY_SHARED_SECRET ?? '';
const BRIEF_TTL_SECONDS = 7 * 24 * 60 * 60; // 7 days
const MAX_STORIES_PER_USER = 12;
const INSIGHTS_KEY = 'news:insights:v1';
// ── Upstash helpers ──────────────────────────────────────────────────────────
/**
* Write the brief envelope via the Upstash REST pipeline endpoint
* (body-POST), not the path-embedded SETEX form. Realistic briefs
* (12 stories, per-story description + whyMatters near caps) encode
* to 520 KB of JSON; URL-encoding inflates that further and can hit
* CDN / edge / Node HTTP request-target limits (commonly 816 KB).
* `redisPipeline` places the command in a JSON body where size
* limits are generous and uniform with the rest of the codebase's
* Upstash writes.
*/
async function upstashSetex(key, value, ttlSeconds) {
const results = await redisPipeline([
['SETEX', key, String(ttlSeconds), JSON.stringify(value)],
]);
if (!results || !Array.isArray(results) || results.length === 0) {
throw new Error(`Upstash SETEX failed for ${key}: null pipeline response`);
}
const result = results[0];
// Upstash pipeline returns either {result} or {error} per command.
if (result && typeof result === 'object' && 'error' in result) {
throw new Error(`Upstash SETEX failed for ${key}: ${result.error}`);
}
return result;
}
// ── Date helpers ─────────────────────────────────────────────────────────────
const MONTH_NAMES = [
'January', 'February', 'March', 'April', 'May', 'June',
'July', 'August', 'September', 'October', 'November', 'December',
];
function dateLongFromIso(iso) {
// iso is YYYY-MM-DD. Parse literally to avoid tz drift.
const [y, m, d] = iso.split('-').map(Number);
return `${d} ${MONTH_NAMES[m - 1]} ${y}`;
}
function issueCodeFromIso(iso) {
// "2026-04-18" → "18.04"
const [, m, d] = iso.split('-');
return `${d}.${m}`;
}
function localHourInTz(nowMs, timezone) {
try {
const fmt = new Intl.DateTimeFormat('en-US', {
timeZone: timezone,
hour: 'numeric',
hour12: false,
});
const hour = fmt.formatToParts(new Date(nowMs))
.find((p) => p.type === 'hour')?.value;
const n = Number(hour);
return Number.isFinite(n) ? n : 9;
} catch {
return 9;
}
}
// ── Convex helpers ───────────────────────────────────────────────────────────
async function fetchDigestRules() {
const res = await fetch(`${CONVEX_SITE_URL}/relay/digest-rules`, {
method: 'GET',
headers: {
Authorization: `Bearer ${RELAY_SECRET}`,
'User-Agent': 'worldmonitor-brief-composer/1.0',
},
signal: AbortSignal.timeout(10_000),
});
if (!res.ok) {
throw new Error(`Failed to fetch digest rules: HTTP ${res.status}`);
}
const rules = await res.json();
if (!Array.isArray(rules)) {
throw new Error('digest-rules response was not an array');
}
return rules;
}
// ── Failure gate ─────────────────────────────────────────────────────────────
/**
* Decide whether the cron should exit non-zero so Railway flags the
* run. Denominator is ATTEMPTED writes (success + failed); skipped-
* empty users never reached the write path and must not inflate it.
* Exported so the denominator contract is testable without mocking
* Redis + LLM + the whole cron.
*
* @param {{ success: number; failed: number; thresholdRatio?: number }} counters
* @returns {boolean}
*/
export function shouldExitNonZero({ success, failed, thresholdRatio = 0.05 }) {
if (failed <= 0) return false;
const attempted = success + failed;
if (attempted <= 0) return false;
const threshold = Math.max(1, Math.floor(attempted * thresholdRatio));
return failed >= threshold;
}
// ── User-name lookup (best effort) ───────────────────────────────────────────
function userDisplayNameFromId(userId) {
// Clerk IDs look like "user_2abc..." — not display-friendly. Phase
// 3b will hydrate names via a Convex query; Phase 3a uses a
// generic "you" so the greeting still reads naturally without a
// round-trip we don't yet need.
void userId;
return 'Reader';
}
// ── Rule dedupe (one brief per user, not per variant) ───────────────────────
// Most-permissive-first ranking. Lower = broader.
const SENSITIVITY_RANK = { all: 0, high: 1, critical: 2 };
function compareRules(a, b) {
// Prefer the 'full' variant — it's the superset dashboard.
const aFull = a.variant === 'full' ? 0 : 1;
const bFull = b.variant === 'full' ? 0 : 1;
if (aFull !== bFull) return aFull - bFull;
// Tie-break on most permissive sensitivity (broadest brief).
const aRank = SENSITIVITY_RANK[a.sensitivity ?? 'all'] ?? 0;
const bRank = SENSITIVITY_RANK[b.sensitivity ?? 'all'] ?? 0;
if (aRank !== bRank) return aRank - bRank;
// Final tie-break: earlier-updated rule wins for determinism.
return (a.updatedAt ?? 0) - (b.updatedAt ?? 0);
}
/**
* Group eligible (non-opted-out) rules by userId, with each user's
* candidates sorted in preference order (best first). Returns an
* array of `[userId, ranked-candidates[]]` pairs so the main loop
* can try each variant in order and fall back when the preferred
* one produces zero stories.
*
* aiDigestEnabled is pre-filtered here so a user whose preferred
* variant is opted out but another variant is opted in still
* produces a brief — the dedupe must not pick a variant that can
* never emit.
*/
export function groupEligibleRulesByUser(rules) {
/** @type {Map<string, any[]>} */
const byUser = new Map();
for (const rule of rules) {
if (!rule || typeof rule.userId !== 'string') continue;
// Default is OPT-IN — only an explicit false opts the user out.
if (rule.aiDigestEnabled === false) continue;
const list = byUser.get(rule.userId);
if (list) list.push(rule);
else byUser.set(rule.userId, [rule]);
}
for (const list of byUser.values()) {
list.sort(compareRules);
}
return byUser;
}
/**
* @deprecated Kept so the existing dedupe tests still compile.
* Prefer groupEligibleRulesByUser + per-user fallback in callers.
*/
export function dedupeRulesByUser(rules) {
const grouped = groupEligibleRulesByUser(rules);
const out = [];
for (const candidates of grouped.values()) {
if (candidates.length > 0) out.push(candidates[0]);
}
return out;
}
// ── Insights fetch ───────────────────────────────────────────────────────────
function extractInsights(raw) {
// news:insights:v1 is stored as a seed envelope {_seed, data}.
// readRawJsonFromUpstash intentionally does not unwrap; do so here.
const data = raw?.data ?? raw;
const topStories = Array.isArray(data?.topStories) ? data.topStories : [];
const clusterCount = Number.isFinite(data?.clusterCount) ? data.clusterCount : topStories.length;
const multiSourceCount = Number.isFinite(data?.multiSourceCount) ? data.multiSourceCount : 0;
return {
topStories,
numbers: {
clusters: clusterCount,
multiSource: multiSourceCount,
},
};
}
// ── SIGTERM handling ─────────────────────────────────────────────────────────
// Matches the bundle-runner SIGTERM pattern (feedback note
// bundle-runner-sigkill-leaks-child-lock). This script does not take
// a distributed lock, but it does perform many parallel Upstash
// writes; SIGTERM during the loop should flush partial progress
// cleanly instead of throwing mid-fetch.
let shuttingDown = false;
process.on('SIGTERM', () => {
shuttingDown = true;
console.log('[brief-composer] SIGTERM received — finishing current iteration');
});
// ── Main ─────────────────────────────────────────────────────────────────────
async function main() {
const startMs = Date.now();
console.log('[brief-composer] Run start:', new Date(startMs).toISOString());
let insightsRaw;
try {
insightsRaw = await readRawJsonFromUpstash(INSIGHTS_KEY);
} catch (err) {
console.error('[brief-composer] failed to read', INSIGHTS_KEY, err.message);
process.exit(1);
}
if (!insightsRaw) {
console.warn('[brief-composer] insights key empty; no brief to compose');
return;
}
const insights = extractInsights(insightsRaw);
if (insights.topStories.length === 0) {
console.warn('[brief-composer] upstream topStories empty; no brief to compose');
return;
}
let rules;
try {
rules = await fetchDigestRules();
} catch (err) {
console.error('[brief-composer]', err.message);
process.exit(1);
}
console.log(`[brief-composer] Rules to process: ${rules.length}`);
// Briefs are user-scoped, but alertRules are (userId, variant)-scoped.
// Group eligible (not-opted-out) rules by user in preference order
// so we can fall back across variants when the preferred one can't
// emit (opt-out on that variant, or zero matching stories).
const eligibleByUser = groupEligibleRulesByUser(rules);
let success = 0;
let skippedEmpty = 0;
let failed = 0;
for (const [userId, candidates] of eligibleByUser) {
if (shuttingDown) break;
try {
// Walk preference order; first variant with non-empty stories wins.
let chosen = null;
let chosenStories = null;
for (const candidate of candidates) {
const sensitivity = candidate.sensitivity ?? 'all';
const stories = filterTopStories({
stories: insights.topStories,
sensitivity,
maxStories: MAX_STORIES_PER_USER,
});
if (stories.length > 0) {
chosen = candidate;
chosenStories = stories;
break;
}
}
if (!chosen) {
skippedEmpty += 1;
continue;
}
if (candidates.length > 1) {
console.log(
`[brief-composer] dedup: userId=${userId} chose variant=${chosen.variant} sensitivity=${chosen.sensitivity ?? 'all'} from ${candidates.length} enabled variants`,
);
}
const tz = chosen.digestTimezone ?? 'UTC';
const issueDate = issueDateInTz(startMs, tz);
const envelope = assembleStubbedBriefEnvelope({
user: { name: userDisplayNameFromId(chosen.userId), tz },
stories: chosenStories,
issueDate,
dateLong: dateLongFromIso(issueDate),
issue: issueCodeFromIso(issueDate),
insightsNumbers: insights.numbers,
issuedAt: Date.now(),
localHour: localHourInTz(startMs, tz),
});
const key = `brief:${chosen.userId}:${issueDate}`;
await upstashSetex(key, envelope, BRIEF_TTL_SECONDS);
success += 1;
} catch (err) {
failed += 1;
const variants = candidates.map((c) => c.variant).join(',');
console.error(
`[brief-composer] failed for user=${userId} variants=${variants}:`,
err.message,
);
}
}
const eligibleUserCount = eligibleByUser.size;
const attempted = success + failed;
const durationMs = Date.now() - startMs;
console.log(
`[brief-composer] Done: rules=${rules.length} eligible_users=${eligibleUserCount} attempted=${attempted} success=${success} skipped_empty=${skippedEmpty} failed=${failed} duration_ms=${durationMs}`,
);
if (shouldExitNonZero({ success, failed })) process.exit(1);
}
// Only run the cron loop when executed as a script, never on import.
// Tests import this file for the dedupe helpers and must not trigger
// process.exit() at module load. Matches feedback_seed_isMain_guard.
function isMain() {
if (!process.argv[1]) return false;
try {
return fileURLToPath(import.meta.url) === process.argv[1];
} catch {
return false;
}
}
if (isMain()) {
if (process.env.BRIEF_COMPOSER_ENABLED === '0') {
console.log('[brief-composer] BRIEF_COMPOSER_ENABLED=0 — skipping run');
process.exit(0);
}
if (!UPSTASH_URL || !UPSTASH_TOKEN) {
console.error('[brief-composer] UPSTASH_REDIS_REST_URL/TOKEN not set');
process.exit(1);
}
if (!CONVEX_SITE_URL || !RELAY_SECRET) {
console.error('[brief-composer] CONVEX_SITE_URL / RELAY_SHARED_SECRET not set');
process.exit(1);
}
main().catch((err) => {
console.error('[brief-composer] fatal:', err);
process.exit(1);
});
}

69
shared/brief-filter.d.ts vendored Normal file
View File

@@ -0,0 +1,69 @@
// Pure helpers for composing a WorldMonitor Brief envelope from the
// upstream news:insights:v1 cache + a user's alert-rule preferences.
//
// Split into its own module so Phase 3a (stubbed digest text) and
// Phase 3b (LLM-generated digest) can share the same filter + shape
// logic. Also importable from tests without pulling in Railway
// runtime deps.
import type {
BriefEnvelope,
BriefStory,
BriefThreatLevel,
} from './brief-envelope.js';
/**
* Upstream `news:insights:v1.topStories[i].threatLevel` uses an
* extended ladder that includes 'moderate' as a synonym for
* 'medium'. Phase 1 of the brief contract pinned the union to four
* values; this helper normalises incoming severities.
*/
export function normaliseThreatLevel(upstream: string): BriefThreatLevel | null;
export type AlertSensitivity = 'all' | 'high' | 'critical';
/**
* Filters the upstream `topStories` array against a user's
* `alertRules.sensitivity` setting and caps at `maxStories`. Stories
* with an unknown upstream severity are dropped.
*/
export function filterTopStories(input: {
stories: UpstreamTopStory[];
sensitivity: AlertSensitivity;
maxStories?: number;
}): BriefStory[];
/**
* Builds a complete BriefEnvelope with stubbed digest text. Phase 3b
* replaces the stubs with LLM output; every other field is final.
*
* Throws if the resulting envelope would fail assertBriefEnvelope —
* the composer never writes an envelope the renderer cannot serve.
*/
export function assembleStubbedBriefEnvelope(input: {
user: { name: string; tz: string };
stories: BriefStory[];
issueDate: string;
dateLong: string;
issue: string;
insightsNumbers: { clusters: number; multiSource: number };
issuedAt?: number;
}): BriefEnvelope;
/**
* Computes the user's local issue date from the current timestamp
* and their IANA timezone. Falls back to UTC today for malformed
* timezones so a composer run never blocks on one bad record.
*/
export function issueDateInTz(nowMs: number, timezone: string): string;
/** Upstream shape from news:insights:v1.topStories[]. */
export interface UpstreamTopStory {
primaryTitle?: unknown;
primarySource?: unknown;
description?: unknown;
threatLevel?: unknown;
category?: unknown;
countryCode?: unknown;
importanceScore?: unknown;
}

236
shared/brief-filter.js Normal file
View File

@@ -0,0 +1,236 @@
// Pure helpers for composing a WorldMonitor Brief envelope from
// upstream news:insights:v1 content + a user's alert-rule preferences.
//
// Split into its own module so Phase 3a (stubbed digest text) and
// Phase 3b (LLM-generated digest) share the same filter + shape
// logic. No I/O, no LLM calls, no network — fully testable.
import { BRIEF_ENVELOPE_VERSION } from './brief-envelope.js';
import { assertBriefEnvelope } from '../server/_shared/brief-render.js';
/**
* @typedef {import('./brief-envelope.js').BriefEnvelope} BriefEnvelope
* @typedef {import('./brief-envelope.js').BriefStory} BriefStory
* @typedef {import('./brief-envelope.js').BriefThreatLevel} BriefThreatLevel
* @typedef {import('./brief-envelope.js').BriefThread} BriefThread
* @typedef {import('./brief-envelope.js').BriefDigest} BriefDigest
* @typedef {import('./brief-filter.js').AlertSensitivity} AlertSensitivity
* @typedef {import('./brief-filter.js').UpstreamTopStory} UpstreamTopStory
*/
// ── Severity normalisation ───────────────────────────────────────────────────
/** @type {Record<string, BriefThreatLevel>} */
const SEVERITY_MAP = {
critical: 'critical',
high: 'high',
medium: 'medium',
// Upstream seed-insights still emits 'moderate' — alias to 'medium'.
moderate: 'medium',
low: 'low',
};
/**
* @param {unknown} upstream
* @returns {BriefThreatLevel | null}
*/
export function normaliseThreatLevel(upstream) {
if (typeof upstream !== 'string') return null;
return SEVERITY_MAP[upstream.toLowerCase()] ?? null;
}
// ── Sensitivity → severity threshold ─────────────────────────────────────────
/** @type {Record<AlertSensitivity, Set<BriefThreatLevel>>} */
const ALLOWED_LEVELS_BY_SENSITIVITY = {
// Matches convex/constants.ts sensitivityValidator: 'all'|'high'|'critical'.
all: new Set(['critical', 'high', 'medium', 'low']),
high: new Set(['critical', 'high']),
critical: new Set(['critical']),
};
// ── Filter ───────────────────────────────────────────────────────────────────
const MAX_HEADLINE_LEN = 200;
const MAX_DESCRIPTION_LEN = 400;
const MAX_SOURCE_LEN = 120;
/** @param {unknown} v */
function asTrimmedString(v) {
if (typeof v !== 'string') return '';
return v.trim();
}
/** @param {string} v @param {number} cap */
function clip(v, cap) {
if (v.length <= cap) return v;
return `${v.slice(0, cap - 1).trimEnd()}\u2026`;
}
/**
* @param {{ stories: UpstreamTopStory[]; sensitivity: AlertSensitivity; maxStories?: number }} input
* @returns {BriefStory[]}
*/
export function filterTopStories({ stories, sensitivity, maxStories = 12 }) {
if (!Array.isArray(stories)) return [];
const allowed = ALLOWED_LEVELS_BY_SENSITIVITY[sensitivity];
if (!allowed) return [];
/** @type {BriefStory[]} */
const out = [];
for (const raw of stories) {
if (out.length >= maxStories) break;
if (!raw || typeof raw !== 'object') continue;
const threatLevel = normaliseThreatLevel(raw.threatLevel);
if (!threatLevel || !allowed.has(threatLevel)) continue;
const headline = clip(asTrimmedString(raw.primaryTitle), MAX_HEADLINE_LEN);
if (!headline) continue;
const description = clip(
asTrimmedString(raw.description) || headline,
MAX_DESCRIPTION_LEN,
);
const source = clip(
asTrimmedString(raw.primarySource) || 'Multiple wires',
MAX_SOURCE_LEN,
);
const category = asTrimmedString(raw.category) || 'General';
const country = asTrimmedString(raw.countryCode) || 'Global';
out.push({
category,
country,
threatLevel,
headline,
description,
source,
// Stubbed at Phase 3a. Phase 3b replaces this with an LLM-
// generated per-user rationale. The renderer requires a non-
// empty string, so we emit a generic fallback rather than
// leaving the field blank.
whyMatters:
'Story flagged by your sensitivity settings. Open for context.',
});
}
return out;
}
// ── Envelope assembly (stubbed digest text) ─────────────────────────────────
function deriveThreadsFromStories(stories) {
const byCategory = new Map();
for (const s of stories) {
const n = byCategory.get(s.category) ?? 0;
byCategory.set(s.category, n + 1);
}
const sorted = [...byCategory.entries()].sort((a, b) => b[1] - a[1]);
return sorted.slice(0, 6).map(([tag, count]) => ({
tag,
teaser:
count === 1
? 'One thread on the desk today.'
: `${count} threads on the desk today.`,
}));
}
function greetingForHour(localHour) {
if (localHour < 5 || localHour >= 22) return 'Good evening.';
if (localHour < 12) return 'Good morning.';
if (localHour < 18) return 'Good afternoon.';
return 'Good evening.';
}
/**
* @param {{
* user: { name: string; tz: string };
* stories: BriefStory[];
* issueDate: string;
* dateLong: string;
* issue: string;
* insightsNumbers: { clusters: number; multiSource: number };
* issuedAt?: number;
* localHour?: number;
* }} input
* @returns {BriefEnvelope}
*/
export function assembleStubbedBriefEnvelope({
user,
stories,
issueDate,
dateLong,
issue,
insightsNumbers,
issuedAt = Date.now(),
localHour,
}) {
const greeting = greetingForHour(
typeof localHour === 'number' ? localHour : 9,
);
/** @type {BriefDigest} */
const digest = {
greeting,
// Phase 3b swaps this with an LLM-generated executive summary.
// Phase 3a uses a neutral placeholder so the magazine still
// renders end-to-end.
lead: `Today's brief surfaces ${stories.length} ${
stories.length === 1 ? 'thread' : 'threads'
} flagged by your sensitivity settings. Open any page to read the full editorial.`,
numbers: {
clusters: insightsNumbers.clusters,
multiSource: insightsNumbers.multiSource,
surfaced: stories.length,
},
threads: deriveThreadsFromStories(stories),
// Signals-to-watch is intentionally empty at Phase 3a. The
// Digest / 04 Signals page is conditional in the renderer, so
// an empty array simply drops that page instead of rendering
// stubbed content that would read as noise.
signals: [],
};
/** @type {BriefEnvelope} */
const envelope = {
version: BRIEF_ENVELOPE_VERSION,
issuedAt,
data: {
user,
issue,
date: issueDate,
dateLong,
digest,
stories,
},
};
// Fail loud if the composer would produce an envelope the
// renderer cannot serve. Phase 1 established this as the central
// contract; drift here is the error mode we most care about.
assertBriefEnvelope(envelope);
return envelope;
}
// ── Tz-aware issue date ──────────────────────────────────────────────────────
/**
* @param {number} nowMs
* @param {string} timezone
* @returns {string}
*/
export function issueDateInTz(nowMs, timezone) {
try {
const fmt = new Intl.DateTimeFormat('en-CA', {
timeZone: timezone,
year: 'numeric',
month: '2-digit',
day: '2-digit',
});
// en-CA conveniently formats as YYYY-MM-DD.
const parts = fmt.format(new Date(nowMs));
if (/^\d{4}-\d{2}-\d{2}$/.test(parts)) return parts;
} catch {
/* fall through to UTC */
}
return new Date(nowMs).toISOString().slice(0, 10);
}

View File

@@ -0,0 +1,204 @@
// Regression tests for the Phase 3a composer's rule-selection logic.
//
// Two guards:
// 1. aiDigestEnabled default parity — undefined must be opt-IN, matching
// seed-digest-notifications.mjs:914 and notifications-settings.ts:228.
// 2. Per-user dedupe — alertRules are (userId, variant)-scoped but the
// brief key is user-scoped. Multi-variant users must produce exactly
// one brief per issue, with a deterministic tie-breaker.
import { describe, it } from 'node:test';
import assert from 'node:assert/strict';
import {
dedupeRulesByUser,
groupEligibleRulesByUser,
shouldExitNonZero,
} from '../scripts/seed-brief-composer.mjs';
function rule(overrides = {}) {
return {
userId: 'user_abc',
variant: 'full',
enabled: true,
digestMode: 'daily',
sensitivity: 'high',
aiDigestEnabled: true,
digestTimezone: 'UTC',
updatedAt: 1_700_000_000_000,
...overrides,
};
}
describe('dedupeRulesByUser', () => {
it('keeps a single rule unchanged', () => {
const out = dedupeRulesByUser([rule()]);
assert.equal(out.length, 1);
assert.equal(out[0].variant, 'full');
});
it('dedupes multi-variant users to one rule, preferring "full"', () => {
const out = dedupeRulesByUser([
rule({ variant: 'finance', sensitivity: 'high' }),
rule({ variant: 'full', sensitivity: 'critical' }),
rule({ variant: 'tech', sensitivity: 'all' }),
]);
assert.equal(out.length, 1);
assert.equal(out[0].variant, 'full');
});
it('when no full variant: picks most permissive sensitivity', () => {
const out = dedupeRulesByUser([
rule({ variant: 'tech', sensitivity: 'critical' }),
rule({ variant: 'finance', sensitivity: 'all' }),
rule({ variant: 'energy', sensitivity: 'high' }),
]);
assert.equal(out.length, 1);
// 'all' is the most permissive.
assert.equal(out[0].variant, 'finance');
});
it('never cross-contaminates across userIds', () => {
const out = dedupeRulesByUser([
rule({ userId: 'user_a', variant: 'full' }),
rule({ userId: 'user_b', variant: 'tech' }),
rule({ userId: 'user_a', variant: 'finance' }),
]);
assert.equal(out.length, 2);
const a = out.find((r) => r.userId === 'user_a');
const b = out.find((r) => r.userId === 'user_b');
assert.equal(a.variant, 'full');
assert.equal(b.variant, 'tech');
});
it('drops rules without a string userId', () => {
const out = dedupeRulesByUser([
rule({ userId: /** @type {any} */ (null) }),
rule({ userId: 'user_ok' }),
]);
assert.equal(out.length, 1);
assert.equal(out[0].userId, 'user_ok');
});
it('is deterministic across duplicate full-variant rules via updatedAt tie-breaker', () => {
const older = rule({ variant: 'full', sensitivity: 'high', updatedAt: 1_000 });
const newer = rule({ variant: 'full', sensitivity: 'high', updatedAt: 2_000 });
const out1 = dedupeRulesByUser([older, newer]);
const out2 = dedupeRulesByUser([newer, older]);
// Earlier updatedAt wins — stable under input reordering.
assert.equal(out1[0].updatedAt, 1_000);
assert.equal(out2[0].updatedAt, 1_000);
});
});
describe('aiDigestEnabled default parity', () => {
// The composer's main loop short-circuits on `rule.aiDigestEnabled
// === false`. Exercising the predicate directly so a refactor that
// re-inverts it (back to `!rule.aiDigestEnabled`) fails loud.
function shouldSkipForAiDigest(rule) {
return rule.aiDigestEnabled === false;
}
it('includes rules with aiDigestEnabled: true', () => {
assert.equal(shouldSkipForAiDigest({ aiDigestEnabled: true }), false);
});
it('includes rules with aiDigestEnabled: undefined (legacy rows)', () => {
assert.equal(shouldSkipForAiDigest({ aiDigestEnabled: undefined }), false);
});
it('includes rules with no aiDigestEnabled field at all (legacy rows)', () => {
assert.equal(shouldSkipForAiDigest({}), false);
});
it('excludes only when explicitly false', () => {
assert.equal(shouldSkipForAiDigest({ aiDigestEnabled: false }), true);
});
it('groupEligibleRulesByUser: opted-out preferred variant falls back to opted-in sibling', () => {
const grouped = groupEligibleRulesByUser([
rule({ variant: 'full', aiDigestEnabled: false, updatedAt: 100 }),
rule({ variant: 'finance', aiDigestEnabled: true, updatedAt: 200 }),
]);
const candidates = grouped.get('user_abc');
assert.ok(candidates, 'user is still eligible via the opt-in variant');
assert.equal(candidates.length, 1);
assert.equal(candidates[0].variant, 'finance');
});
it('groupEligibleRulesByUser: user with all variants opted-out is dropped entirely', () => {
const grouped = groupEligibleRulesByUser([
rule({ variant: 'full', aiDigestEnabled: false }),
rule({ variant: 'finance', aiDigestEnabled: false }),
]);
assert.equal(grouped.size, 0);
});
it('groupEligibleRulesByUser: retains all eligible candidates in preference order', () => {
const grouped = groupEligibleRulesByUser([
rule({ variant: 'finance', sensitivity: 'critical', updatedAt: 100 }),
rule({ variant: 'full', sensitivity: 'critical', updatedAt: 200 }),
rule({ variant: 'tech', sensitivity: 'all', updatedAt: 300 }),
]);
const candidates = grouped.get('user_abc');
assert.equal(candidates.length, 3);
// First is full (preferred variant); then tech (most permissive sensitivity);
// then finance. Fallback loop in the main() script tries them in this order.
assert.equal(candidates[0].variant, 'full');
assert.equal(candidates[1].variant, 'tech');
assert.equal(candidates[2].variant, 'finance');
});
it('shouldExitNonZero: returns false when no failures', () => {
assert.equal(shouldExitNonZero({ success: 10, failed: 0 }), false);
});
it('shouldExitNonZero: catches 100% failure on small attempted volume', () => {
// 4 attempted, 4 failed, 96 eligible skipped-empty. The earlier
// (eligibleUserCount) denominator would read 4/100=4% and pass.
assert.equal(shouldExitNonZero({ success: 0, failed: 4 }), true);
});
it('shouldExitNonZero: 1/20 failures is exactly at 5% (floor(20*0.05)=1), trips', () => {
// Exact-threshold boundary: documents intentional behaviour.
assert.equal(shouldExitNonZero({ success: 19, failed: 1 }), true);
});
it('shouldExitNonZero: 1/50 failures stays under threshold (floor(50*0.05)=2)', () => {
// Threshold floor is Math.max(1, floor(N*0.05)). For N<40 a
// single failure always trips. At N=50 the threshold is 2, so
// 1/50 stays green. Ops intuition: the 5% bar is only a "bar"
// once you have a meaningful sample.
assert.equal(shouldExitNonZero({ success: 49, failed: 1 }), false);
});
it('shouldExitNonZero: 2/10 exceeds threshold', () => {
// floor(10 * 0.05) = 0 → Math.max forces 1. failed=2 >= 1.
assert.equal(shouldExitNonZero({ success: 8, failed: 2 }), true);
});
it('shouldExitNonZero: single isolated failure still tripwires', () => {
// floor(1 * 0.05) = 0 → Math.max forces 1. failed=1 >= 1.
assert.equal(shouldExitNonZero({ success: 0, failed: 1 }), true);
});
it('shouldExitNonZero: zero attempted means no signal, returns false', () => {
assert.equal(shouldExitNonZero({ success: 0, failed: 0 }), false);
});
it('matches seed-digest-notifications convention', async () => {
// Cross-reference: the existing digest cron uses the same
// `!== false` test. If it drifts, the brief and digest will
// disagree on who is eligible. This assertion lives here to
// surface the divergence loudly.
const fs = await import('node:fs/promises');
const src = await fs.readFile(
new URL('../scripts/seed-digest-notifications.mjs', import.meta.url),
'utf8',
);
assert.ok(
src.includes('rule.aiDigestEnabled !== false'),
'seed-digest-notifications.mjs must keep `rule.aiDigestEnabled !== false`',
);
});
});

227
tests/brief-filter.test.mjs Normal file
View File

@@ -0,0 +1,227 @@
// Pure-function tests for the Phase 3a brief composer helpers.
//
// Locks in: severity normalisation (moderate → medium), sensitivity
// threshold, story cap, envelope assembly passes the renderer's
// strict validator, threads derivation, tz-aware issue date.
import { describe, it } from 'node:test';
import assert from 'node:assert/strict';
import {
normaliseThreatLevel,
filterTopStories,
assembleStubbedBriefEnvelope,
issueDateInTz,
} from '../shared/brief-filter.js';
import { BRIEF_ENVELOPE_VERSION } from '../shared/brief-envelope.js';
function upstreamStory(overrides = {}) {
return {
primaryTitle: 'Iran declares Strait of Hormuz open. Oil drops more than 9%.',
primarySource: 'Reuters',
description: 'Tehran publicly reopened the Strait of Hormuz to commercial shipping today.',
threatLevel: 'high',
category: 'Energy',
countryCode: 'IR',
importanceScore: 320,
...overrides,
};
}
describe('normaliseThreatLevel', () => {
it('accepts the four canonical values', () => {
for (const level of ['critical', 'high', 'medium', 'low']) {
assert.equal(normaliseThreatLevel(level), level);
}
});
it('maps upstream "moderate" to "medium"', () => {
assert.equal(normaliseThreatLevel('moderate'), 'medium');
});
it('case-insensitive', () => {
assert.equal(normaliseThreatLevel('HIGH'), 'high');
assert.equal(normaliseThreatLevel('Moderate'), 'medium');
});
it('returns null on unknown or non-string input', () => {
assert.equal(normaliseThreatLevel('unknown'), null);
assert.equal(normaliseThreatLevel(null), null);
assert.equal(normaliseThreatLevel(42), null);
});
});
describe('filterTopStories', () => {
it('respects sensitivity=critical (keeps critical only)', () => {
const out = filterTopStories({
stories: [
upstreamStory({ threatLevel: 'critical' }),
upstreamStory({ threatLevel: 'high' }),
upstreamStory({ threatLevel: 'medium' }),
],
sensitivity: 'critical',
});
assert.equal(out.length, 1);
assert.equal(out[0].threatLevel, 'critical');
});
it('sensitivity=high keeps critical + high', () => {
const out = filterTopStories({
stories: [
upstreamStory({ threatLevel: 'critical' }),
upstreamStory({ threatLevel: 'high' }),
upstreamStory({ threatLevel: 'medium' }),
upstreamStory({ threatLevel: 'low' }),
],
sensitivity: 'high',
});
assert.equal(out.length, 2);
});
it('sensitivity=all keeps everything with a known severity', () => {
const out = filterTopStories({
stories: [
upstreamStory({ threatLevel: 'critical' }),
upstreamStory({ threatLevel: 'high' }),
upstreamStory({ threatLevel: 'moderate' }),
upstreamStory({ threatLevel: 'low' }),
upstreamStory({ threatLevel: 'unknown' }),
],
sensitivity: 'all',
});
assert.equal(out.length, 4);
});
it('caps at maxStories', () => {
const stories = Array.from({ length: 20 }, (_, i) =>
upstreamStory({ primaryTitle: `Story ${i}` }),
);
const out = filterTopStories({ stories, sensitivity: 'all', maxStories: 5 });
assert.equal(out.length, 5);
});
it('falls back to Multiple wires when primarySource missing', () => {
const out = filterTopStories({
stories: [upstreamStory({ primarySource: '' })],
sensitivity: 'all',
});
assert.equal(out[0].source, 'Multiple wires');
});
it('drops stories with empty primaryTitle', () => {
const out = filterTopStories({
stories: [upstreamStory({ primaryTitle: ' ' })],
sensitivity: 'all',
});
assert.equal(out.length, 0);
});
it('returns empty for unknown sensitivity', () => {
const out = filterTopStories({
stories: [upstreamStory()],
sensitivity: /** @type {any} */ ('bogus'),
});
assert.equal(out.length, 0);
});
it('non-array input returns empty', () => {
assert.deepEqual(
filterTopStories({
stories: /** @type {any} */ (null),
sensitivity: 'all',
}),
[],
);
});
});
describe('assembleStubbedBriefEnvelope', () => {
const baseStories = [
upstreamStory({ threatLevel: 'critical' }),
upstreamStory({ threatLevel: 'high', category: 'Diplomacy' }),
upstreamStory({ threatLevel: 'high', category: 'Maritime' }),
upstreamStory({ threatLevel: 'medium', category: 'Energy' }),
];
function baseInput() {
const stories = filterTopStories({
stories: baseStories,
sensitivity: 'all',
});
return {
user: { name: 'Elie', tz: 'UTC' },
stories,
issueDate: '2026-04-18',
dateLong: '18 April 2026',
issue: '18.04',
insightsNumbers: { clusters: 278, multiSource: 21 },
issuedAt: 1_700_000_000_000,
localHour: 9,
};
}
it('produces an envelope that passes the strict renderer validator', () => {
const env = assembleStubbedBriefEnvelope(baseInput());
assert.equal(env.version, BRIEF_ENVELOPE_VERSION);
assert.equal(env.data.digest.numbers.surfaced, env.data.stories.length);
assert.equal(env.data.digest.signals.length, 0);
assert.ok(env.data.digest.threads.length > 0);
});
it('morning greeting at hour 9', () => {
const env = assembleStubbedBriefEnvelope({ ...baseInput(), localHour: 9 });
assert.equal(env.data.digest.greeting, 'Good morning.');
});
it('evening greeting at hour 22', () => {
const env = assembleStubbedBriefEnvelope({ ...baseInput(), localHour: 22 });
assert.equal(env.data.digest.greeting, 'Good evening.');
});
it('afternoon greeting at hour 14', () => {
const env = assembleStubbedBriefEnvelope({ ...baseInput(), localHour: 14 });
assert.equal(env.data.digest.greeting, 'Good afternoon.');
});
it('threads are derived from category frequency, capped at 6', () => {
const many = Array.from({ length: 10 }, (_, i) =>
upstreamStory({ category: `Cat${i}`, threatLevel: 'high' }),
);
const stories = filterTopStories({ stories: many, sensitivity: 'all' });
const env = assembleStubbedBriefEnvelope({
...baseInput(),
stories,
});
assert.ok(env.data.digest.threads.length <= 6);
});
it('throws when assembled envelope would fail validation (empty stories)', () => {
assert.throws(() =>
assembleStubbedBriefEnvelope({
...baseInput(),
stories: [],
}),
);
});
});
describe('issueDateInTz', () => {
// 2026-04-18T00:30:00Z — midnight UTC + 30min. Tokyo (+9) is
// already mid-morning on the 18th; LA (-7) is late on the 17th.
const midnightUtc = Date.UTC(2026, 3, 18, 0, 30, 0);
it('UTC returns the UTC date', () => {
assert.equal(issueDateInTz(midnightUtc, 'UTC'), '2026-04-18');
});
it('positive offset (Asia/Tokyo) returns the later local date', () => {
assert.equal(issueDateInTz(midnightUtc, 'Asia/Tokyo'), '2026-04-18');
});
it('negative offset (America/Los_Angeles) returns the earlier local date', () => {
assert.equal(issueDateInTz(midnightUtc, 'America/Los_Angeles'), '2026-04-17');
});
it('malformed timezone falls back to UTC', () => {
assert.equal(issueDateInTz(midnightUtc, 'Not/A_Zone'), '2026-04-18');
});
});