feat(e3): story persistence tracking (#2620)

Browse Source

* feat(e3): story persistence tracking

Adds cross-cycle story tracking layer to the RSS digest pipeline:

- Proto: StoryMeta message + StoryPhase enum on NewsItem (fields 9-11).
  importanceScore and corroborationCount stubs added for E1.
- list-feed-digest.ts: builds corroboration map across ALL items before
  truncation; batch-reads existing story:track hashes from Redis; writes
  HINCRBY/HSET/HSETNX/SADD/EXPIRE per story in 80-story pipeline chunks;
  attaches StoryMeta (firstSeen, mentionCount, sourceCount, phase) to
  each proto item using read-back data.
- cache-keys.ts: STORY_TRACK_KEY_PREFIX, STORY_SOURCES_KEY_PREFIX,
  DIGEST_ACCUMULATOR_KEY_PREFIX, STORY_TRACKING_TTL_S.
- src/types/index.ts: StoryMeta, StoryPhase, NewsItem extended.
- data-loader.ts: protoItemToNewsItem maps STORY_PHASE_* → client phase.
- NewsPanel.ts: BREAKING/DEVELOPING/ONGOING phase badges in item rows.

New story first appearance: phase=BREAKING. After 2 mentions within 2h:
DEVELOPING. After 6+ mentions or >2h: SUSTAINED. If score drops below
50% of peak: FADING (used by E1; defaults to SUSTAINED for now).

Redis keys per story (48h TTL):
  story:track:v1:<hash16>   → hash (firstSeen,lastSeen,mentionCount,...)
  story:sources:v1:<hash16> → set  (feed names, for cross-source count)

* fix(e3): correct storyMeta staleness and mentionCount semantics

P1 — storyMeta was always one cycle behind because storyTracks was read
before writeStoryTracking ran. Fix: keep read-before-write but compute
storyMeta from merged in-memory state (stale.mentionCount + 1, fresh
sourceCount from corroborationMap). New stories get mentionCount=1 and
phase=BREAKING in the same cycle they first appear — no extra Redis
round-trip needed.

P2 — mentionCount incremented once per item occurrence, so a story seen
in 3 sources in its first cycle was immediately stored as mentionCount=3.
Fix: deduplicate by titleHash in writeStoryTracking so each unique story
gets exactly one HINCRBY per digest cycle regardless of source count.
SADD still collects all sources for the set key.

* fix(e3): Unicode hash collision, ALERT badge regression, FADING comment

P1 — normalizeTitle used [^\w\s] without the u flag; \w is ASCII-only
so every Arabic/CJK/Cyrillic title stripped to "" and shared one Redis
hash. Fixed: use /[^\p{L}\p{N}\s]/gu (Unicode property escapes require
the u flag).

P1 — ALERT badge was gated on !item.storyMeta, suppressing the indicator
for any tracked story regardless of isAlert. Phase and alert are
orthogonal signals; ALERT now renders unconditionally when isAlert=true.

P2 — FADING branch is intentionally inactive until E1 ships real scores
(currentScore/peakScore placeholder 0 via HSETNX). Added comment to
document the intentional ordering.

* fix(news-alerts): skip sustained/fading stories in breaking alert selectBest

Sustained and fading story phases are already well-covered by the feed;
only breaking and developing phases warrant a banner interrupt. Items
without storyMeta (phase unspecified) pass through unchanged.

Fixes gap C from docs/plans/2026-04-02-003-fix-news-alerts-pr-gaps-plan.md

* fix(e3): remove rebase artifacts from list-feed-digest

Removes a stray closing brace, duplicate ASCII normalizeTitle
(Unicode-aware version from the fix commit is correct), and
a leftover storyPhase assignment that references a removed field.

All typecheck and typecheck:api pass clean.

...

This commit is contained in:

Elie Habib

2026-04-02 21:16:35 +04:00

committed by GitHub

parent 72a9801402

commit 6c017998d3

11 changed files with 217 additions and 100 deletions

Show all changes Ignore whitespace when comparing lines Ignore changes in amount of whitespace Ignore changes in whitespace at EOL

Download Patch File Download Diff File Expand all files Collapse all files

2

docs/api/NewsService.openapi.json

File diff suppressed because one or more lines are too long