mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
80cb7d5aa7
* fix(cache): align Redis digest + RSS feed TTLs to CF CDN TTL
RSS feed TTL 600s → 3600s; digest TTL 900s → 3600s.
CF CDN caches at 3600s, so Redis expiring earlier caused every hourly
CF revalidation to hit a cold origin and run the full buildDigest()
pipeline (75 feeds, up to 25s). Aligning both to 3600s ensures CF
revalidation gets a warm Redis hit and returns immediately.
* fix(cache): emit only non-ok feedStatuses; update proto comment + make generate
Digest was emitting 'ok' for every successful feed (~50 entries, ~1-2KB
per response). No in-repo client reads feedStatuses values. Changed to
only emit 'empty' and 'timeout'; absent key implies ok.
Updated proto comment to document the absence-implies-ok contract and
ran make generate to regenerate docs/api/ OpenAPI files.
* fix(cache): add slow-browser tier; move digest route to it
New 'slow-browser' tier is identical to 'slow' but adds max-age=300,
letting browsers skip the network for 5 minutes. Without max-age,
browsers ignore s-maxage and send conditional If-None-Match on every
20-min poll — each costing 1 billable edge request even for 304s.
Scoped only to list-feed-digest (a safe polling endpoint). Premium
user-triggered endpoints (analyze-stock, backtest-stock) stay on 'slow'
where browser caching is inappropriate.
* test: regression tests for feedStatuses and slow-browser tier
- digest-no-reclassify: assert buildDigest does not write 'ok' to feedStatuses
- route-cache-tier: include slow-browser in tier regex; assert slow-browser
has max-age and slow tier does not
* fix(cache): add variant to per-feed RSS cache key
rss:feed:v1:${url} was shared across variants even though classifyByKeyword()
bakes variant-specific threat/category labels into the cached ParsedItem[].
Feeds shared between full and tech variants (Verge, Ars, HN, etc.) had
whichever variant populated the cache first control the other variant's
classifications for the full 3600s TTL — turning a pre-existing 10-minute
bleed-through into a 1-hour accuracy bug for the tech dashboard.
Fix: key is now rss:feed:v1:${variant}:${url}.
* fix(cache): bypass browser HTTP cache on digest fetch
max-age=300 on the slow-browser tier lets browsers serve the digest
from their HTTP cache for up to 5 minutes, including on explicit
in-app refresh (window.location.reload) or page reload after a
breaking event. Users would see stale data until the TTL expired.
Add cache: 'no-cache' to tryFetchDigest() so every fetch revalidates
against CF edge. CF returns 304 (minimal cost) when data is unchanged,
or 200 with the current digest. s-maxage and CF-level caching are
unaffected; max-age still benefits browser back/forward cache.
* fix(cache): 15-min consistent TTL + degrade guard for digest
Issue 1 — TTL alignment: Redis digest TTL reverted to 900s (from 3600).
slow-browser tier reduced from s-maxage=1800/CDN=3600 to s-maxage=900 on
both sides, matching the Redis TTL. The freshness window is now consistently
15 minutes across Redis, Vercel edge, and CF CDN. max-age=300 (browser
local) is kept to avoid unnecessary revalidations on tab switch.
Issue 2 — Cache poisoning: replaced cachedFetchJson in listFeedDigest with
explicit getCachedJson/setCachedJson. After buildDigest(), if total items
across all categories is 0 the response is treated as degraded: Redis write
is skipped and markNoCacheResponse(ctx.request) is called so the gateway
sets Cache-Control: no-store instead of the normal tier headers. This
prevents a transient bad run from poisoning Redis and browser/CDN for the
full TTL. Error paths also call markNoCacheResponse.
66 lines
2.7 KiB
JavaScript
66 lines
2.7 KiB
JavaScript
/**
|
|
* Regression test: digest-backed news items must NOT trigger client-side
|
|
* classifyWithAI calls. The server digest already runs enrichWithAiCache()
|
|
* against the same Redis keys, so client reclassification wastes edge requests.
|
|
*
|
|
* Run: node --test tests/digest-no-reclassify.test.mjs
|
|
*/
|
|
|
|
import { describe, it } from 'node:test';
|
|
import assert from 'node:assert/strict';
|
|
import { readFileSync } from 'node:fs';
|
|
import { resolve, dirname } from 'node:path';
|
|
import { fileURLToPath } from 'node:url';
|
|
|
|
const __dirname = dirname(fileURLToPath(import.meta.url));
|
|
const src = readFileSync(resolve(__dirname, '..', 'src', 'app', 'data-loader.ts'), 'utf-8');
|
|
const serverSrc = readFileSync(
|
|
resolve(__dirname, '..', 'server', 'worldmonitor', 'news', 'v1', 'list-feed-digest.ts'),
|
|
'utf-8',
|
|
);
|
|
|
|
describe('Digest branch must not reclassify with AI', () => {
|
|
const digestBranchStart = src.indexOf("// Digest branch: server already aggregated feeds");
|
|
const digestBranchEnd = src.indexOf('} else {', digestBranchStart);
|
|
const digestBranch = src.slice(digestBranchStart, digestBranchEnd);
|
|
|
|
it('digest branch exists in data-loader.ts', () => {
|
|
assert.ok(digestBranchStart !== -1, 'Digest branch comment must exist');
|
|
assert.ok(digestBranchEnd > digestBranchStart, 'Digest branch must have an else clause');
|
|
});
|
|
|
|
it('digest branch does NOT call classifyWithAI', () => {
|
|
assert.ok(!digestBranch.includes('classifyWithAI'),
|
|
'Digest items must not trigger classifyWithAI (server already classified via enrichWithAiCache)');
|
|
});
|
|
|
|
it('digest branch does NOT call canQueueAiClassification', () => {
|
|
assert.ok(!digestBranch.includes('canQueueAiClassification'),
|
|
'Digest items must not be queued for AI classification');
|
|
});
|
|
|
|
it('digest branch does NOT reference aiCandidates', () => {
|
|
assert.ok(!digestBranch.includes('aiCandidates'),
|
|
'No aiCandidates filtering should exist in the digest branch');
|
|
});
|
|
|
|
it('classifyWithAI is not imported in data-loader.ts', () => {
|
|
assert.ok(!src.includes("import { classifyWithAI }") && !src.includes("import { classifyWithAI,"),
|
|
'classifyWithAI should not be imported (no call sites remain)');
|
|
});
|
|
|
|
it('canQueueAiClassification is not imported in data-loader.ts', () => {
|
|
assert.ok(!src.includes("import { canQueueAiClassification"),
|
|
'canQueueAiClassification should not be imported (no call sites remain)');
|
|
});
|
|
});
|
|
|
|
describe('feedStatuses must not emit ok entries', () => {
|
|
it('buildDigest does not write ok to feedStatuses', () => {
|
|
assert.ok(
|
|
!serverSrc.includes("feedStatuses[feed.name] = items.length > 0 ? 'ok' : 'empty'"),
|
|
"feedStatuses must not write 'ok' entries — wastes payload on every response",
|
|
);
|
|
});
|
|
});
|