Files
worldmonitor/scripts/lib/brief-compose.mjs
Elie Habib 711636c7b6 feat(brief): consolidate composer into digest cron (retire standalone service) (#3157)
* feat(brief): consolidate composer into digest cron (retire standalone service)

Merges the Phase 3a standalone Railway composer into the existing
digest cron. End state: one cron (seed-digest-notifications.mjs)
writes brief:{userId}:{issueDate} for every eligible user AND
dispatches the digest to their configured channels with a signed
magazine URL appended. Net -1 Railway service.

User's architectural note: "there is no reason to have 1 digest
preparing all and sending, then another doing a duplicate". This
delivers that — infrastructure consolidation, same send cadence,
single source of truth for brief envelopes.

File moves / deletes:

- scripts/seed-brief-composer.mjs → scripts/lib/brief-compose.mjs
  Pure-helpers library: no main(), no env guards, no cron. Exports
  composeBriefForRule + groupEligibleRulesByUser + dedupeRulesByUser
  (shim) + shouldExitNonZero + date helpers + extractInsights.
- Dockerfile.seed-brief-composer → deleted.
- The seed-brief-composer Railway service is retired (user confirmed
  they would delete it manually).

New files:

- scripts/lib/brief-url-sign.mjs — plain .mjs port of the sign path
  in server/_shared/brief-url.ts (Web Crypto only, no node:crypto).
- tests/brief-url-sign.test.mjs — parity tests that confirm tokens
  minted by the scripts-side signer verify via the edge-side verifier
  and produce byte-identical output for identical input.

Digest cron (scripts/seed-digest-notifications.mjs):

- Reads news:insights:v1 once per run, composes per-user brief
  envelopes, SETEX brief:{userId}:{issueDate} via body-POST pipeline.
- Signs magazine URL per user (BRIEF_URL_SIGNING_SECRET +
  WORLDMONITOR_PUBLIC_BASE_URL new env requirements, see pre-merge).
- Injects magazineUrl into buildChannelBodies for every channel
  (email, telegram, slack, discord) as a "📖 Open your WorldMonitor
  Brief magazine" footer CTA.
- Email HTML gets a dedicated data-brief-cta-slot near the top of
  the body with a styled button.
- Compose failures NEVER block the digest send — the digest cron's
  existing behaviour is preserved when the brief pipeline has issues.
- Brief compose extracted to its own functions (composeBriefsForRun
  + composeAndStoreBriefForUser) to keep main's biome complexity at
  baseline (64 — was 63 before; inline would have pushed to 117).

Tests: 98/98 across the brief suite. New parity tests confirm cross-
module signer agreement.

PRE-MERGE: add BRIEF_URL_SIGNING_SECRET and WORLDMONITOR_PUBLIC_BASE_URL
to the digest-notifications Railway service env (same values already
set on Vercel for Phase 2). Without them, brief compose is auto-
disabled and the digest falls back to its current behaviour — safe to
deploy before env is set.

* fix(brief): digest Dockerfile + propagate compose failure to exit code

Addresses two seventh-round review findings on PR #3157.

1. Cross-directory imports + current Railway build root (todo 230).
   The consolidated digest cron imports from ../api, ../shared, and
   (transitively via scripts/lib/brief-compose.mjs) ../server/_shared.
   The running digest-notifications Railway service builds from the
   scripts/ root — those parent paths are outside the deploy tree
   and would 500 on next rebuild with ERR_MODULE_NOT_FOUND.

   New Dockerfile.digest-notifications (repo-root build context)
   COPYs exactly the modules the cron needs: scripts/ contents,
   scripts/lib/, shared/brief-envelope.*, shared/brief-filter.*,
   server/_shared/brief-render.*, api/_upstash-json.js,
   api/_seed-envelope.js. Tight list to keep the watch surface small.
   Pattern matches the retired Dockerfile.seed-brief-composer + the
   existing Dockerfile.relay.

2. Silent compose failures (todo 231). composeBriefsForRun logged
   counters but never exited non-zero. An Upstash outage or missing
   signing secret silently dropped every brief write while Railway
   showed the cron green. The retired standalone composer exited 1
   on structural failures; that observability was lost in the
   consolidation.

   Changed the compose fn to return {briefByUser, composeSuccess,
   composeFailed}. Main captures the counters, runs the full digest
   send loop first (compose-layer breakage must NEVER block user-
   visible digest delivery), then calls shouldExitNonZero at the
   very end. Exit-on-failure gives ops the Railway-red signal
   without touching send behaviour.

   Also: a total read failure of news:insights:v1 (catch branch)
   now counts as 1 compose failure so the gate trips on insights-
   key infra breakage, not just per-user write failures.

Tests unchanged (98/98). Typecheck + node --check clean. Biome
complexity ticks 63→65 — same pre-existing bucket, already tolerated
by CI; no new blocker.

PRE-MERGE Railway work still pending: set BRIEF_URL_SIGNING_SECRET
+ WORLDMONITOR_PUBLIC_BASE_URL on the digest-notifications service,
AND switch its dockerfilePath to /Dockerfile.digest-notifications
before merging. Without the dockerfilePath switch, the next rebuild
fails.

* fix(brief): Dockerfile type:module + explicit missing-secret tripwire

Addresses two eighth-round review findings on PR #3157.

1. ESM .js files parse as CommonJS in the container (todo 232).
   Dockerfile.digest-notifications COPYs shared/*.js,
   server/_shared/*.js, api/*.js — all ESM because the repo-root
   package.json has "type":"module". But the image never copies the
   root package.json, so Node's nearest-pjson walk inside /app/
   reaches / without finding one and defaults to CommonJS. First
   `export` statement throws `SyntaxError: Unexpected token 'export'`
   at startup.

   Fix: write a minimal /app/package.json with {"type":"module"}
   early in the build. Avoids dragging the full root package.json
   into the image while still giving Node the ESM hint it needs for
   repo-owned .js files.

2. Missing BRIEF_URL_SIGNING_SECRET silently tolerated (todo 233).
   The old gate folded "operator-disabled" (BRIEF_COMPOSE_ENABLED=0)
   and "required secret missing in rollout" into the same boolean
   via AND. A production deploy that forgot the env var would skip
   brief compose without any failure signal — Railway green, no
   briefs, no CTA in digests, nobody notices.

   Split the two states: BRIEF_COMPOSE_DISABLED_BY_OPERATOR (explicit
   kill switch, silent) and BRIEF_SIGNING_SECRET_MISSING (the misconfig
   we care about). When the secret is missing without the operator
   flag, composeBriefsForRun returns composeFailed=1 on first call
   so the end-of-run exit gate trips and Railway flags the run red.
   Digest send still proceeds — compose-layer issues never block
   notifications.

Tests: 98/98. Syntax + node --check clean.

* fix(brief): address 2 remaining P2 review comments on PR #3157

Greptile review (2026-04-18T05:04Z) flagged three P2 items. The
first (shouldExitNonZero never wired into cron) was already fixed in
commit 35a46aa34. This commit addresses the other two.

1. composeBriefForRule: issuedAt used Date.now() instead of the
   caller-supplied nowMs. Under the digest cron the delta is
   milliseconds and harmless, but it broke the function's
   determinism contract — same input must produce same output for
   tests + retries. Now uses the passed nowMs.

2. buildChannelBodies: magazineUrl embedded raw inside Telegram HTML
   <a href="..."> and Slack <URL|text> syntax. The URL is HMAC-
   signed and shape-validated upstream (userId regex + YYYY-MM-DD
   date), so injection is practically impossible — but the email
   CTA (injectBriefCta) escapes per-target and channel footers
   should match that discipline. Added:
     - Telegram: escape &, <, >, " to HTML entities
     - Slack: strip <, >, | (mrkdwn metacharacters)
   Discord and plain-text paths unchanged — Discord links tolerate
   raw URLs, plain text has no metacharacters to escape.

Tests: 98/98 still pass (deterministic issuedAt change was
transparent to existing assertions because tests already pass nowMs
explicitly via the issuedAt fixture field).
2026-04-18 12:30:08 +04:00

182 lines
6.7 KiB
JavaScript

// WorldMonitor Brief compose library.
//
// Pure helpers for producing the per-user brief envelope that the
// hosted magazine route (api/brief/*) + dashboard panel + future
// channels all consume. Shared between:
// - scripts/seed-digest-notifications.mjs (the consolidated cron;
// composes a brief for every user it's about to dispatch a
// digest to, so the magazine URL can be injected into the
// notification output).
// - future tests + ad-hoc tools.
//
// Deliberately has NO top-level side effects: no env guards, no
// process.exit, no main(). Import anywhere.
//
// History: this file used to include a stand-alone Railway cron
// (`seed-brief-composer.mjs`). That path was retired in the
// consolidation PR — the digest cron now owns the compose+send
// pipeline so there is exactly one cron writing brief:{userId}:
// {issueDate} keys.
import {
assembleStubbedBriefEnvelope,
filterTopStories,
issueDateInTz,
} from '../../shared/brief-filter.js';
// ── Rule dedupe (one brief per user, not per variant) ───────────────────────
const SENSITIVITY_RANK = { all: 0, high: 1, critical: 2 };
function compareRules(a, b) {
const aFull = a.variant === 'full' ? 0 : 1;
const bFull = b.variant === 'full' ? 0 : 1;
if (aFull !== bFull) return aFull - bFull;
const aRank = SENSITIVITY_RANK[a.sensitivity ?? 'all'] ?? 0;
const bRank = SENSITIVITY_RANK[b.sensitivity ?? 'all'] ?? 0;
if (aRank !== bRank) return aRank - bRank;
return (a.updatedAt ?? 0) - (b.updatedAt ?? 0);
}
/**
* Group eligible (not-opted-out) rules by userId with each user's
* candidates sorted in preference order. Callers walk the candidate
* list and take the first that produces non-empty stories — falls
* back across variants cleanly.
*/
export function groupEligibleRulesByUser(rules) {
const byUser = new Map();
for (const rule of rules) {
if (!rule || typeof rule.userId !== 'string') continue;
if (rule.aiDigestEnabled === false) continue;
const list = byUser.get(rule.userId);
if (list) list.push(rule);
else byUser.set(rule.userId, [rule]);
}
for (const list of byUser.values()) list.sort(compareRules);
return byUser;
}
/**
* @deprecated Kept for existing test imports. Prefer
* groupEligibleRulesByUser + per-user fallback at call sites.
*/
export function dedupeRulesByUser(rules) {
const out = [];
for (const candidates of groupEligibleRulesByUser(rules).values()) {
if (candidates.length > 0) out.push(candidates[0]);
}
return out;
}
// ── Failure gate ─────────────────────────────────────────────────────────────
/**
* Decide whether the consolidated cron should exit non-zero because
* the brief-write failure rate is structurally bad (not just a
* transient blip). Denominator is ATTEMPTED writes, not eligible
* users: skipped-empty users never reach the write path and must not
* dilute the ratio.
*
* @param {{ success: number; failed: number; thresholdRatio?: number }} counters
*/
export function shouldExitNonZero({ success, failed, thresholdRatio = 0.05 }) {
if (failed <= 0) return false;
const attempted = success + failed;
if (attempted <= 0) return false;
const threshold = Math.max(1, Math.floor(attempted * thresholdRatio));
return failed >= threshold;
}
// ── Insights fetch ───────────────────────────────────────────────────────────
/** Unwrap news:insights:v1 envelope and project the fields the brief needs. */
export function extractInsights(raw) {
const data = raw?.data ?? raw;
const topStories = Array.isArray(data?.topStories) ? data.topStories : [];
const clusterCount = Number.isFinite(data?.clusterCount) ? data.clusterCount : topStories.length;
const multiSourceCount = Number.isFinite(data?.multiSourceCount) ? data.multiSourceCount : 0;
return {
topStories,
numbers: { clusters: clusterCount, multiSource: multiSourceCount },
};
}
// ── Date + display helpers ───────────────────────────────────────────────────
const MONTH_NAMES = [
'January', 'February', 'March', 'April', 'May', 'June',
'July', 'August', 'September', 'October', 'November', 'December',
];
export function dateLongFromIso(iso) {
const [y, m, d] = iso.split('-').map(Number);
return `${d} ${MONTH_NAMES[m - 1]} ${y}`;
}
export function issueCodeFromIso(iso) {
const [, m, d] = iso.split('-');
return `${d}.${m}`;
}
export function localHourInTz(nowMs, timezone) {
try {
const fmt = new Intl.DateTimeFormat('en-US', {
timeZone: timezone,
hour: 'numeric',
hour12: false,
});
const hour = fmt.formatToParts(new Date(nowMs)).find((p) => p.type === 'hour')?.value;
const n = Number(hour);
return Number.isFinite(n) ? n : 9;
} catch {
return 9;
}
}
export function userDisplayNameFromId(userId) {
// Clerk IDs look like "user_2abc…". Phase 3b will hydrate real
// names via a Convex query; for now a generic placeholder so the
// magazine's greeting reads naturally.
void userId;
return 'Reader';
}
// ── Compose a full brief for a single rule ──────────────────────────────────
const MAX_STORIES_PER_USER = 12;
/**
* Filter + assemble a BriefEnvelope for one alert rule. Returns null
* when the filter produces zero stories — the caller decides whether
* to fall back to another variant or skip the user.
*
* @param {object} rule — enabled alertRule row
* @param {{ topStories: unknown[]; numbers: { clusters: number; multiSource: number } }} insights
* @param {{ nowMs: number }} [opts]
*/
export function composeBriefForRule(rule, insights, { nowMs = Date.now() } = {}) {
const sensitivity = rule.sensitivity ?? 'all';
const tz = rule.digestTimezone ?? 'UTC';
const stories = filterTopStories({
stories: insights.topStories,
sensitivity,
maxStories: MAX_STORIES_PER_USER,
});
if (stories.length === 0) return null;
const issueDate = issueDateInTz(nowMs, tz);
return assembleStubbedBriefEnvelope({
user: { name: userDisplayNameFromId(rule.userId), tz },
stories,
issueDate,
dateLong: dateLongFromIso(issueDate),
issue: issueCodeFromIso(issueDate),
insightsNumbers: insights.numbers,
// Same nowMs as the rest of the envelope so the function stays
// deterministic for a given input — tests + retries see identical
// output.
issuedAt: nowMs,
localHour: localHourInTz(nowMs, tz),
});
}