mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
feat(brief): make MAX_STORIES_PER_USER env-tunable (default 12, evidence kept it at 12) (#3389)
* fix(brief): bump MAX_STORIES_PER_USER 12 → 16 Production telemetry from PR #3387 surfaced cap-truncation as the dominant filter loss: 73% of `sensitivity=all` users had `dropped_cap=18` per tick (30 qualified stories truncated to 12). Multi-member topics straddling the position-12 boundary lost members. Bumping the cap to 16 lets larger leading topics fit fully without affecting `sensitivity=critical` users (their pools cap at 7-10 stories — well below either threshold). Reduces dropped_cap from ~18 to ~14 per tick. Validation signal: watch the `[digest] brief filter drops` log line on Railway after deploy — `dropped_cap=` should drop by ~4 per tick. Side effect: this addresses the dominant production signal that Solution 3 (post-filter regroup, originally planned in docs/plans/2026-04-24-004-fix-brief-topic-adjacency-defects-plan.md) was supposed to handle. Production evidence killed Sol-3's premise (0 non-cap drops in 70 samples), so this is a simpler, evidence-backed alternative. * revise(brief): keep MAX_STORIES_PER_USER default at 12, add env-tunability Reviewer asked "why 16?" and the honest answer turned out to be: the data doesn't support it. After landing PR #3390's sweep harness with visible-window metrics, re-ran against 2026-04-24 production replay: threshold=0.45 cap=12 -> visible_quality 0.916 (best at this cap) threshold=0.45 cap=16 -> visible_quality 0.716 (cap bump HURTS) threshold=0.42 cap=12 -> visible_quality 0.845 threshold=0.42 cap=16 -> visible_quality 0.845 (neutral) At the current 0.45 threshold, positions 13-16 are mostly singletons or members of "should-separate" clusters — they dilute the brief without helping topic adjacency. Bumping the cap default to 16 was a wrong inference from the dropped_cap=18 signal alone. Revised approach: - Default MAX_STORIES_PER_USER stays at 12 (matches historical prod). - Constant becomes env-tunable via DIGEST_MAX_STORIES_PER_USER so any future sweep result can be acted on with a Railway env flip without a redeploy. The actual evidence-backed adjacency fix from the sweep is to lower DIGEST_DEDUP_TOPIC_THRESHOLD from 0.45 -> 0.42 (env flip; see PR #3390). * fix(brief-llm): tie buildDigestPrompt + hashDigestInput slice to MAX_STORIES_PER_USER Greptile P1 on PR #3389: with MAX_STORIES_PER_USER now env-tunable, hard-coded stories.slice(0, 12) in buildDigestPrompt and hashDigestInput would mean the LLM prose only references the first 12 stories when the brief carries more. Stories 13+ would appear as visible cards but be invisible to the AI summary — a quiet mismatch between reader narrative and brief content. Cache key MUST stay aligned with the prompt slice or it drifts from the prompt content; same constant fixes both sites. Exports MAX_STORIES_PER_USER from brief-compose.mjs (single source of truth) and imports it in brief-llm.mjs. No behaviour change at the default cap of 12.
This commit is contained in:
@@ -192,7 +192,14 @@ describe('composeBriefFromDigestStories — continued', () => {
|
||||
assert.deepEqual(env.data.stories.map((s) => s.headline), ['A', 'B']);
|
||||
});
|
||||
|
||||
it('caps at 12 stories per brief', () => {
|
||||
it('caps at 12 stories per brief by default (env-tunable via DIGEST_MAX_STORIES_PER_USER)', () => {
|
||||
// Default kept at 12. Offline sweep harness against 2026-04-24
|
||||
// production replay showed cap=16 dropped visible_quality from
|
||||
// 0.916 → 0.716 at the active 0.45 threshold (positions 13-16
|
||||
// are mostly singletons or "should-separate" members at this
|
||||
// threshold, so they dilute without helping adjacency). The
|
||||
// constant is env-tunable so a Railway flip can experiment with
|
||||
// cap values once new sweep evidence justifies them.
|
||||
const many = Array.from({ length: 30 }, (_, i) =>
|
||||
digestStory({ hash: `h${i}`, title: `Story ${i}` }),
|
||||
);
|
||||
|
||||
Reference in New Issue
Block a user