mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
chore(api): sebuf migration follow-ups (post-#3242) (#3287)
* chore(api-manifest): rewrite brief-why-matters reason as proper internal-helper justification Carried in from #3248 merge as a band-aid (called out in #3242 review followup checklist item 7). The endpoint genuinely belongs in internal-helper — RELAY_SHARED_SECRET-bearer auth, cron-only caller, never reached by dashboards or partners. Same shape constraint as api/notify.ts. Replaces the apologetic "filed here to keep the lint green" framing with a proper structural justification: modeling it as a generated service would publish internal cron plumbing as user-facing API surface. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat(lint): premium-fetch parity check for ServiceClients (closes #3279) Adds scripts/enforce-premium-fetch.mjs — AST-walks src/, finds every `new <ServiceClient>(...)` (variable decl OR `this.foo =` assignment), tracks which methods each instance actually calls, and fails if any called method targets a path in src/shared/premium-paths.ts PREMIUM_RPC_PATHS without `{ fetch: premiumFetch }` on the constructor. Per-call-site analysis (not class-level) keeps the trade/index.ts pattern clean — publicClient with globalThis.fetch + premiumClient with premiumFetch on the same TradeServiceClient class — since publicClient never calls a premium method. Wired into: - npm run lint:premium-fetch - .husky/pre-push (right after lint:rate-limit-policies) - .github/workflows/lint-code.yml (right after lint:api-contract) Found and fixed three latent instances of the HIGH(new) #1 class from #3242 review (silent 401 → empty fallback for signed-in browser pros): - src/services/correlation-engine/engine.ts — IntelligenceServiceClient built with no fetch option called deductSituation. LLM-assessment overlay on convergence cards never landed for browser pros without a WM key. - src/services/economic/index.ts — EconomicServiceClient with globalThis.fetch called getNationalDebt. National-debt panel rendered empty for browser pros. - src/services/sanctions-pressure.ts — SanctionsServiceClient with globalThis.fetch called listSanctionsPressure. Sanctions-pressure panel rendered empty for browser pros. All three swap to premiumFetch (single shared client, mirrors the supply-chain/index.ts justification — premiumFetch no-ops safely on public methods, so the public methods on those clients keep working). Verification: - lint:premium-fetch clean (34 ServiceClient classes, 28 premium paths, 466 src/ files analyzed) - Negative test: revert any of the three to globalThis.fetch → exit 1 with file:line and called-premium-method names - typecheck + typecheck:api clean - lint:api-contract / lint:rate-limit-policies / lint:boundaries clean - tests/sanctions-pressure.test.mjs + premium-fetch.test.mts: 16/16 pass Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(military): fetchStaleFallback NEG_TTL=30s parity (closes #3277) The legacy /api/military-flights handler had NEG_TTL = 30_000ms — a short suppression window after a failed live + stale read so we don't Redis-hammer the stale key during sustained relay+seed outages. Carried into the sebuf list-military-flights handler: - Module-scoped `staleNegUntil` timestamp (per-isolate on Vercel Edge, which is fine — each warm isolate gets its own 30s suppression window). - Set whenever fetchStaleFallback returns null (key missing, parse fail, empty array after staleToProto filter, or thrown error). - Checked at the entry of fetchStaleFallback before doing the Redis read. - Test seam `_resetStaleNegativeCacheForTests()` exposed for unit tests. Test pinned in tests/redis-caching.test.mjs: drives a stale-empty cycle three times — first read hits Redis, second within window doesn't, after test-only reset it does again. Verified: 18/18 redis-caching tests pass, typecheck:api clean, lint:premium-fetch clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lint): rate-limit-policies regex → import() (closes #3278) The previous lint regex-parsed ENDPOINT_RATE_POLICIES from the source file. That worked because the literal happens to fit a single line per key today, but a future reformat (multi-line key wrap, formatter swap, etc.) would silently break the lint without breaking the build — exactly the failure mode that's worse than no lint at all. Fix: - Export ENDPOINT_RATE_POLICIES from server/_shared/rate-limit.ts. - Convert scripts/enforce-rate-limit-policies.mjs to async + dynamic import() of the policy object directly. Same TS module that the gateway uses at runtime → no source-of-truth drift possible. - Run via tsx (already a dev dep, used by test:data) so the .mjs shebang can resolve a .ts import. - npm script swapped to `tsx scripts/...`. .husky/pre-push uses `npm run lint:rate-limit-policies` so no hook change needed. Verified: - Clean: 6 policies / 182 gateway routes. - Negative test (rename a key to the original sanctions typo /api/sanctions/v1/lookup-entity): exit 1 with the same incident- attributed remedy message as before. - Reformat test (split a single-line entry across multiple lines): still passes — the property is what's read, not the source layout. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(shipping/v2): alertThreshold: 0 preserved; drop dead validation branch (#3242 followup) Before: alert_threshold was a plain int32. proto3 scalar default is 0, so the handler couldn't distinguish "partner explicitly sent 0 (deliver every disruption)" from "partner omitted the field (apply legacy default 50)" — both arrived as 0 and got coerced to 50 by `> 0 ? : 50`. Silent intent-drop for any partner who wanted every alert. The subsequent `alertThreshold < 0` branch was also unreachable after that coercion. After: - Proto field is `optional int32 alert_threshold` — TS type becomes `alertThreshold?: number`, so omitted = undefined and explicit 0 stays 0. - Handler uses `req.alertThreshold ?? 50` — undefined → 50, any number passes through unchanged. - Dead `< 0 || > 100` runtime check removed; buf.validate `int32.gte = 0, int32.lte = 100` already enforces the range at the wire layer. Partner wire contract: identical for the omit-field and 1..100 cases. Only behavioural change is explicit 0 — previously impossible to request, now honored per proto3 optional semantics. Scoped `buf generate --path worldmonitor/shipping/v2` to avoid the full- regen `@ts-nocheck` drift Seb documented in the #3242 PR comments. Re-applied `@ts-nocheck` on the two regenerated files manually. Tests: - `alertThreshold 0 coerces to 50` flipped to `alertThreshold 0 preserved`. - New test: `alertThreshold omitted (undefined) applies legacy default 50`. - `rejects > 100` test removed — proto/wire validation handles it; direct handler calls intentionally bypass wire and the handler no longer carries a redundant runtime range check. Verified: 18/18 shipping-v2-handler tests pass, typecheck + typecheck:api clean, all 4 custom lints clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs(shipping/v2): document missing webhook delivery worker + DNS-rebinding contract (#3242 followup) #3242 followup checklist item 6 from @koala73 — sanity-check that the delivery worker honors the re-resolve-and-re-check contract that isBlockedCallbackUrl explicitly delegates to it. Audit finding: no delivery worker for shipping/v2 webhooks exists in this repo. Grep across the entire tree (excluding generated/dist) shows the only readers of webhook:sub:* records are the registration / inspection / rotate-secret handlers themselves. No code reads them and POSTs to the stored callbackUrl. The delivery worker is presumed to live in Railway (separate repo) or hasn't been built yet — neither is auditable from this repo. Refreshes the comment block at the top of webhook-shared.ts to: - explicitly state DNS rebinding is NOT mitigated at registration - spell out the four-step contract the delivery worker MUST follow (re-validate URL, dns.lookup, re-check resolved IP against patterns, fetch with resolved IP + Host header preserved) - flag the in-repo gap so anyone landing delivery code can't miss it Tracking the gap as #3288 — acceptance there is "delivery worker imports the patterns + helpers from webhook-shared.ts and applies the four steps before each send." Action moves to wherever the delivery worker actually lives (Railway likely). No code change. Tests + lints unchanged. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * ci(lint): add rate-limit-policies step (greptile P1 #3287) Pre-push hook ran lint:rate-limit-policies but the CI workflow did not, so fork PRs and --no-verify pushes bypassed the exact drift check the lint was added to enforce (closes #3278). Adding it right after lint:api-contract so it runs in the same context the lint was designed for. * refactor(lint): premium-fetch regex → import() + loop classRe (greptile P2 #3287) Two fragilities greptile flagged on enforce-premium-fetch.mjs: 1. loadPremiumPaths regex-parsed src/shared/premium-paths.ts with /'(\/api\/[^']+)'/g — same class of silent drift we just removed from enforce-rate-limit-policies in #3278. Reformatting the source Set (double quotes, spread, helper-computed entries) would drop paths from the lint while leaving the runtime untouched. Fix: flip the shebang to `#!/usr/bin/env -S npx tsx` and dynamic-import PREMIUM_RPC_PATHS directly, mirroring the rate-limit pattern. package.json lint:premium-fetch now invokes via tsx too so the npm-script path matches direct execution. 2. loadClientClassMap ran classRe.exec once, silently dropping every ServiceClient after the first if a file ever contained more than one. Current codegen emits one class per file so this was latent, but a template change would ship un-linted classes. Fix: collect every class-open match with matchAll, slice each class body with the next class's start as the boundary, and scan methods per-body so method-to-class binding stays correct even with multiple classes per file. Verification: - lint:premium-fetch clean (34 classes / 28 premium paths / 466 files — identical counts to pre-refactor, so no coverage regression). - Negative test: revert src/services/economic/index.ts to globalThis.fetch → exit 1 with file:line, bound var name, and premium method list (getNationalDebt). Restore → clean. - lint:rate-limit-policies still clean. * fix(shipping/v2): re-add alertThreshold handler range guard (greptile nit 1 #3287) Wire-layer buf.validate enforces 0..100, but direct handler invocation (internal jobs, test harnesses, future transports) bypasses it. Cheap invariant-at-the-boundary — rejects < 0 or > 100 with ValidationError before the record is stored. Tests: restored the rejects-out-of-range cases that were dropped when the branch was (correctly) deleted as dead code on the previous commit. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lint): premium-fetch method-regex → TS AST (greptile nits 2+5 #3287) loadClientClassMap: The method regex `async (\w+)\s*\([^)]*\)\s*:\s*Promise<[^>]+>\s*\{\s*let path = "..."` assumed (a) no nested `)` in arg types, (b) no nested `>` in the return type, (c) `let path = "..."` as the literal first statement. Any codegen template shift would silently drop methods with the lint still passing clean — the same silent-drift class #3287 just closed on the premium-paths side. Now walks the service_client.ts AST, matches `export class *ServiceClient`, iterates `MethodDeclaration` members, and reads the first `let path: string = '...'` variable statement as a StringLiteral. Tolerant to any reformatting of arg/return types or method shape. findCalls scope-blindness: Added limitation comment — the walker matches `<varName>.<method>()` anywhere in the file without respecting scope. Two constructions in different function scopes sharing a var name merge their called-method sets. No current src/ file hits this; the lint errs cautiously (flags both instances). Keeping the walker simple until scope-aware binding is needed. webhook-shared.ts: Inlined issue reference (#3288) so the breadcrumb resolves without bouncing through an MDX that isn't in the diff. Verification: - lint:premium-fetch clean — 34 classes / 28 premium paths / 489 files. Pre-refactor: 34 / 28 / 466. Class + path counts identical; file bump is from the main-branch rebase, not the refactor. - Negative test: revert src/services/economic/index.ts premiumFetch → globalThis.fetch. Lint exits 1 at `src/services/economic/index.ts:64:7` with `premium method(s) called: getNationalDebt`. Restore → clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * refactor(lint): rate-limit OpenAPI regex → yaml parser (greptile nit 3 #3287) Input side (ENDPOINT_RATE_POLICIES) was flipped to live `import()` in4e79d029. Output side (OpenAPI routes) still regex-scraped top-level `paths:` keys with `/^\s{4}(\/api\/[^\s:]+):/gm` — hard-coded 4-space indent. Any YAML formatter change (2-space indent, flow style, line folding) would silently drop routes and let policy-drift slip through — same silent-drift class the input-side fix closed. Now uses the `yaml` package (already a dep) to parse each .openapi.yaml and reads `doc.paths` directly. Verification: - Clean: 6 policies / 189 routes (was 182 — yaml parser picks up a handful the regex missed, closing a silent coverage gap). - Negative test: rename policy key back to /api/sanctions/v1/lookup-entity → exits 1 with the same incident-attributed remedy. Restore → clean. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore(codegen): regenerate unified OpenAPI bundle for alert_threshold proto change The shipping/v2 webhook alert_threshold field was flipped from `int32` to `optional int32` with an expanded doc comment inf3339464. That comment now surfaces in the unified docs/api/worldmonitor.openapi.yaml bundle (introduced by #3341). Regenerated with sebuf v0.11.1 to pick it up. No behaviour change — bundle-only documentation drift. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
2
.github/workflows/lint-code.yml
vendored
2
.github/workflows/lint-code.yml
vendored
@@ -44,6 +44,8 @@ jobs:
|
||||
- run: npm run lint
|
||||
- run: npm run lint:boundaries
|
||||
- run: npm run lint:api-contract
|
||||
- run: npm run lint:rate-limit-policies
|
||||
- run: npm run lint:premium-fetch
|
||||
- name: Markdown lint
|
||||
run: npm run lint:md
|
||||
- name: Version sync check
|
||||
|
||||
@@ -68,6 +68,9 @@ npm run lint:boundaries || exit 1
|
||||
echo "Running rate-limit policy coverage check..."
|
||||
npm run lint:rate-limit-policies || exit 1
|
||||
|
||||
echo "Running premium-fetch parity check..."
|
||||
npm run lint:premium-fetch || exit 1
|
||||
|
||||
echo "Running edge function bundle check..."
|
||||
while IFS= read -r f; do
|
||||
npx esbuild "$f" --bundle --format=esm --platform=browser --outfile=/dev/null 2>/dev/null || {
|
||||
|
||||
@@ -247,7 +247,7 @@
|
||||
{
|
||||
"path": "api/internal/brief-why-matters.ts",
|
||||
"category": "internal-helper",
|
||||
"reason": "Internal brief-pipeline helper — auth'd by RELAY_SHARED_SECRET (Railway cron only), not a user-facing API. Generated on merge of #3248 from main without a manifest entry; filed here to keep the lint green.",
|
||||
"reason": "LLM-enrichment helper for the brief pipeline. Cron-triggered from Railway under RELAY_SHARED_SECRET bearer auth — never reached by dashboards, desktop, or partners. Request/response shape is an implementation detail of the brief renderer; modeling it as a generated service would publish internal cron plumbing as user-facing API surface. Same shape constraint as api/notify.ts (also cron-only).",
|
||||
"owner": "@SebastienMelki",
|
||||
"removal_issue": null
|
||||
},
|
||||
|
||||
File diff suppressed because one or more lines are too long
@@ -277,7 +277,12 @@ components:
|
||||
maximum: 100
|
||||
minimum: 0
|
||||
format: int32
|
||||
description: Disruption-score threshold for delivery, 0-100. Default 50.
|
||||
description: |-
|
||||
Disruption-score threshold for delivery, 0-100. Default 50.
|
||||
proto3 `optional` so the handler can distinguish "partner explicitly sent
|
||||
0 (deliver every alert)" from "partner omitted the field (apply default
|
||||
50)". Without `optional`, both serialise to the proto3 scalar default of
|
||||
0 and the handler can't tell them apart — flagged in #3242 review.
|
||||
required:
|
||||
- callbackUrl
|
||||
description: |-
|
||||
|
||||
@@ -18929,7 +18929,12 @@ components:
|
||||
maximum: 100
|
||||
minimum: 0
|
||||
format: int32
|
||||
description: Disruption-score threshold for delivery, 0-100. Default 50.
|
||||
description: |-
|
||||
Disruption-score threshold for delivery, 0-100. Default 50.
|
||||
proto3 `optional` so the handler can distinguish "partner explicitly sent
|
||||
0 (deliver every alert)" from "partner omitted the field (apply default
|
||||
50)". Without `optional`, both serialise to the proto3 scalar default of
|
||||
0 and the handler can't tell them apart — flagged in #3242 review.
|
||||
required:
|
||||
- callbackUrl
|
||||
description: |-
|
||||
|
||||
@@ -9,7 +9,8 @@
|
||||
"lint:fix": "biome check ./src ./server ./api ./tests ./e2e ./scripts ./middleware.ts --fix",
|
||||
"lint:boundaries": "node scripts/lint-boundaries.mjs",
|
||||
"lint:api-contract": "node scripts/enforce-sebuf-api-contract.mjs",
|
||||
"lint:rate-limit-policies": "node scripts/enforce-rate-limit-policies.mjs",
|
||||
"lint:rate-limit-policies": "tsx scripts/enforce-rate-limit-policies.mjs",
|
||||
"lint:premium-fetch": "tsx scripts/enforce-premium-fetch.mjs",
|
||||
"lint:unicode": "node scripts/check-unicode-safety.mjs",
|
||||
"lint:unicode:staged": "node scripts/check-unicode-safety.mjs --staged",
|
||||
"lint:md": "markdownlint-cli2 '**/*.md' '!**/node_modules/**' '!.agent/**' '!.agents/**' '!.claude/**' '!.factory/**' '!.windsurf/**' '!skills/**' '!docs/internal/**' '!docs/Docs_To_Review/**' '!todos/**' '!docs/plans/**' '!docs/brainstorms/**' '!docs/ideation/**'",
|
||||
|
||||
@@ -20,7 +20,11 @@ message RegisterWebhookRequest {
|
||||
// the entire CHOKEPOINT_REGISTRY. Unknown IDs fail with 400.
|
||||
repeated string chokepoint_ids = 2;
|
||||
// Disruption-score threshold for delivery, 0-100. Default 50.
|
||||
int32 alert_threshold = 3 [
|
||||
// proto3 `optional` so the handler can distinguish "partner explicitly sent
|
||||
// 0 (deliver every alert)" from "partner omitted the field (apply default
|
||||
// 50)". Without `optional`, both serialise to the proto3 scalar default of
|
||||
// 0 and the handler can't tell them apart — flagged in #3242 review.
|
||||
optional int32 alert_threshold = 3 [
|
||||
(buf.validate.field).int32.gte = 0,
|
||||
(buf.validate.field).int32.lte = 100
|
||||
];
|
||||
|
||||
282
scripts/enforce-premium-fetch.mjs
Normal file
282
scripts/enforce-premium-fetch.mjs
Normal file
@@ -0,0 +1,282 @@
|
||||
#!/usr/bin/env -S npx tsx
|
||||
/**
|
||||
* Validates that every `new <ServiceClient>(...)` instantiation in src/ which
|
||||
* calls a method whose generated path is in PREMIUM_RPC_PATHS is constructed
|
||||
* with `{ fetch: premiumFetch }`.
|
||||
*
|
||||
* Catches the HIGH(new) #1 class from #3242 review — SupplyChainServiceClient
|
||||
* was constructed with globalThis.fetch (the generated default) and pro users
|
||||
* silently got 401s the generated client swallowed into empty-fallback panels.
|
||||
* Same class as #3233 (RegionalIntelligenceBoard / DeductionPanel / trade /
|
||||
* country-intel) which was fixed manually because there was no enforcement.
|
||||
*
|
||||
* How it works:
|
||||
* 1. Dynamic `import()` of PREMIUM_RPC_PATHS from src/shared/premium-paths.ts
|
||||
* (via tsx, same pattern as enforce-rate-limit-policies.mjs) → set of
|
||||
* premium HTTP paths. Live import means reformatting the source literal
|
||||
* can never desync the lint from the runtime (#3287 follow-up).
|
||||
* 2. Walk src/generated/client/ → map each ServiceClient class to its
|
||||
* method-name → path table (the `let path = "/api/..."` line each
|
||||
* generated method opens with).
|
||||
* 3. Walk src/ (excluding generated) with the TypeScript AST. For each
|
||||
* `new <ClassName>(...)` (variable decl OR `this.foo =` assignment):
|
||||
* a. Capture the bound variable / member name.
|
||||
* b. Find every `<varName>.<method>(...)` call in the same file.
|
||||
* c. If any called method has a premium path, the construction MUST
|
||||
* use { fetch: premiumFetch }. Anything else fails the lint.
|
||||
*
|
||||
* Per-call-site analysis lets the trade/index.ts pattern (publicClient with
|
||||
* globalThis.fetch + premiumClient with premiumFetch on the same class)
|
||||
* stay clean, since publicClient never calls a premium method.
|
||||
*/
|
||||
import { readFileSync, readdirSync, statSync } from 'node:fs';
|
||||
import { join, relative, basename } from 'node:path';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
import ts from 'typescript';
|
||||
|
||||
const ROOT = new URL('..', import.meta.url).pathname;
|
||||
const PREMIUM_PATHS_SRC = join(ROOT, 'src/shared/premium-paths.ts');
|
||||
const GEN_CLIENT_DIR = join(ROOT, 'src/generated/client');
|
||||
const SRC_DIR = join(ROOT, 'src');
|
||||
|
||||
function walk(dir, fn) {
|
||||
for (const name of readdirSync(dir)) {
|
||||
const full = join(dir, name);
|
||||
const s = statSync(full);
|
||||
if (s.isDirectory()) walk(full, fn);
|
||||
else if (s.isFile()) fn(full);
|
||||
}
|
||||
}
|
||||
|
||||
async function loadPremiumPaths() {
|
||||
// Dynamic import via file URL — runs under tsx (the shebang) which
|
||||
// transparently transpiles TS. Importing the live Set means any reformat of
|
||||
// the source literal (single→double quotes, spread, helper-computed entries)
|
||||
// can never desync the lint from the runtime.
|
||||
const mod = await import(pathToFileURL(PREMIUM_PATHS_SRC).href);
|
||||
if (!(mod.PREMIUM_RPC_PATHS instanceof Set) || mod.PREMIUM_RPC_PATHS.size === 0) {
|
||||
throw new Error(
|
||||
`${PREMIUM_PATHS_SRC} must export PREMIUM_RPC_PATHS as a non-empty Set<string> — the lint relies on it.`,
|
||||
);
|
||||
}
|
||||
return mod.PREMIUM_RPC_PATHS;
|
||||
}
|
||||
|
||||
function loadClientClassMap() {
|
||||
// AST walk rather than regex — the earlier regex
|
||||
// /async (\w+)\s*\([^)]*\)\s*:\s*Promise<[^>]+>\s*\{\s*let path = "([^"]+)"/
|
||||
// assumed (a) no nested `)` in arg types, (b) no nested `>` in the return
|
||||
// type, (c) `let path = "..."` as the literal first statement. Any shift in
|
||||
// the codegen template would silently drop methods and the lint would pass
|
||||
// clean with missing coverage — the same silent-drift class this PR closed
|
||||
// on the premium-paths side (#3287 greptile nit 2).
|
||||
const map = new Map();
|
||||
walk(GEN_CLIENT_DIR, (file) => {
|
||||
if (basename(file) !== 'service_client.ts') return;
|
||||
const src = readFileSync(file, 'utf8');
|
||||
const ast = ts.createSourceFile(file, src, ts.ScriptTarget.Latest, true);
|
||||
|
||||
function visit(node) {
|
||||
if (
|
||||
ts.isClassDeclaration(node) &&
|
||||
node.name &&
|
||||
/ServiceClient$/.test(node.name.text) &&
|
||||
node.modifiers?.some((m) => m.kind === ts.SyntaxKind.ExportKeyword)
|
||||
) {
|
||||
const methods = new Map();
|
||||
for (const member of node.members) {
|
||||
if (!ts.isMethodDeclaration(member)) continue;
|
||||
if (!member.name || !ts.isIdentifier(member.name)) continue;
|
||||
const methodName = member.name.text;
|
||||
const body = member.body;
|
||||
if (!body) continue;
|
||||
// Look for the first `let path = "/api/..."` variable statement in
|
||||
// the method body. Generated clients open each RPC method with it.
|
||||
for (const stmt of body.statements) {
|
||||
if (!ts.isVariableStatement(stmt)) continue;
|
||||
const decl = stmt.declarationList.declarations[0];
|
||||
if (
|
||||
decl &&
|
||||
ts.isIdentifier(decl.name) &&
|
||||
decl.name.text === 'path' &&
|
||||
decl.initializer &&
|
||||
ts.isStringLiteral(decl.initializer)
|
||||
) {
|
||||
methods.set(methodName, decl.initializer.text);
|
||||
break;
|
||||
}
|
||||
}
|
||||
}
|
||||
map.set(node.name.text, methods);
|
||||
}
|
||||
ts.forEachChild(node, visit);
|
||||
}
|
||||
visit(ast);
|
||||
});
|
||||
if (map.size === 0) {
|
||||
throw new Error(`No ServiceClient classes parsed from ${GEN_CLIENT_DIR}`);
|
||||
}
|
||||
return map;
|
||||
}
|
||||
|
||||
function collectSourceFiles() {
|
||||
const out = [];
|
||||
walk(SRC_DIR, (file) => {
|
||||
if (file.startsWith(GEN_CLIENT_DIR)) return;
|
||||
if (!/\.(ts|tsx)$/.test(file)) return;
|
||||
if (file.endsWith('.d.ts')) return;
|
||||
out.push(file);
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
function getFetchOptionText(optionsArg) {
|
||||
if (!optionsArg) return null;
|
||||
if (!ts.isObjectLiteralExpression(optionsArg)) return optionsArg.getText();
|
||||
for (const prop of optionsArg.properties) {
|
||||
if (!ts.isPropertyAssignment(prop)) continue;
|
||||
const name = prop.name && ts.isIdentifier(prop.name) ? prop.name.text : null;
|
||||
if (name === 'fetch') return prop.initializer.getText();
|
||||
}
|
||||
return null;
|
||||
}
|
||||
|
||||
function checkFile(filePath, clientClassMap, premiumPaths) {
|
||||
const src = readFileSync(filePath, 'utf8');
|
||||
const ast = ts.createSourceFile(filePath, src, ts.ScriptTarget.Latest, true);
|
||||
|
||||
const instances = [];
|
||||
|
||||
function recordInstance(varName, newExpr, posNode) {
|
||||
const className = newExpr.expression.getText();
|
||||
if (!clientClassMap.has(className)) return;
|
||||
const optionsArg = newExpr.arguments?.[1] ?? null;
|
||||
const lc = ast.getLineAndCharacterOfPosition(posNode.getStart());
|
||||
instances.push({
|
||||
varName,
|
||||
className,
|
||||
optionsArg,
|
||||
line: lc.line + 1,
|
||||
column: lc.character + 1,
|
||||
});
|
||||
}
|
||||
|
||||
function visit(node) {
|
||||
if (
|
||||
ts.isVariableDeclaration(node) &&
|
||||
node.initializer &&
|
||||
ts.isNewExpression(node.initializer) &&
|
||||
ts.isIdentifier(node.name)
|
||||
) {
|
||||
recordInstance(node.name.text, node.initializer, node);
|
||||
} else if (
|
||||
ts.isBinaryExpression(node) &&
|
||||
node.operatorToken.kind === ts.SyntaxKind.EqualsToken &&
|
||||
ts.isNewExpression(node.right)
|
||||
) {
|
||||
const lhs = node.left.getText();
|
||||
recordInstance(lhs, node.right, node);
|
||||
}
|
||||
ts.forEachChild(node, visit);
|
||||
}
|
||||
visit(ast);
|
||||
|
||||
if (instances.length === 0) return [];
|
||||
|
||||
const violations = [];
|
||||
for (const inst of instances) {
|
||||
const methods = clientClassMap.get(inst.className);
|
||||
const calledMethods = new Set();
|
||||
|
||||
// Scope-blind walk — matches any `<varName>.<method>()` anywhere in the
|
||||
// file. If two constructions in different function scopes share the same
|
||||
// variable name (e.g. both declare `const client = new XServiceClient()`
|
||||
// in unrelated functions), their called-method sets merge and the lint
|
||||
// errs on the side of caution (flags premium calls against both
|
||||
// instances). No current src/ file hits this — keeping the walker
|
||||
// simple until scope-aware binding is actually needed (#3287 nit 5).
|
||||
function findCalls(node) {
|
||||
if (
|
||||
ts.isCallExpression(node) &&
|
||||
ts.isPropertyAccessExpression(node.expression)
|
||||
) {
|
||||
const objText = node.expression.expression.getText();
|
||||
const methodName = node.expression.name.text;
|
||||
if (objText === inst.varName) calledMethods.add(methodName);
|
||||
}
|
||||
ts.forEachChild(node, findCalls);
|
||||
}
|
||||
findCalls(ast);
|
||||
|
||||
const premiumCalls = [...calledMethods].filter((m) => {
|
||||
const path = methods.get(m);
|
||||
return path && premiumPaths.has(path);
|
||||
});
|
||||
if (premiumCalls.length === 0) continue;
|
||||
|
||||
const fetchText = getFetchOptionText(inst.optionsArg);
|
||||
if (fetchText === 'premiumFetch') continue;
|
||||
|
||||
violations.push({
|
||||
file: filePath,
|
||||
line: inst.line,
|
||||
column: inst.column,
|
||||
varName: inst.varName,
|
||||
className: inst.className,
|
||||
fetchText: fetchText ?? '<no fetch option — defaults to globalThis.fetch>',
|
||||
premiumCalls,
|
||||
});
|
||||
}
|
||||
return violations;
|
||||
}
|
||||
|
||||
async function main() {
|
||||
const premiumPaths = await loadPremiumPaths();
|
||||
const clientClassMap = loadClientClassMap();
|
||||
const files = collectSourceFiles();
|
||||
|
||||
const violations = [];
|
||||
for (const f of files) {
|
||||
violations.push(...checkFile(f, clientClassMap, premiumPaths));
|
||||
}
|
||||
|
||||
if (violations.length > 0) {
|
||||
console.error(
|
||||
`\u2717 ${violations.length} ServiceClient instantiation(s) call PREMIUM_RPC_PATHS methods without { fetch: premiumFetch }:\n`,
|
||||
);
|
||||
for (const v of violations) {
|
||||
const rel = relative(ROOT, v.file);
|
||||
console.error(` ${rel}:${v.line}:${v.column}`);
|
||||
console.error(` new ${v.className}(...) bound to \`${v.varName}\``);
|
||||
console.error(` fetch option: ${v.fetchText}`);
|
||||
console.error(` premium method(s) called: ${v.premiumCalls.join(', ')}`);
|
||||
console.error('');
|
||||
}
|
||||
console.error('Each ServiceClient that calls a method whose path is in');
|
||||
console.error('src/shared/premium-paths.ts PREMIUM_RPC_PATHS must be constructed with');
|
||||
console.error(' { fetch: premiumFetch }');
|
||||
console.error('imported from @/services/premium-fetch.\n');
|
||||
console.error('Why: globalThis.fetch sends no auth header, so signed-in browser pros');
|
||||
console.error('without a WORLDMONITOR_API_KEY get a 401 the generated client swallows');
|
||||
console.error('into the empty fallback. premiumFetch injects WM key / Clerk bearer when');
|
||||
console.error('available and no-ops safely otherwise — safe to use even on a client whose');
|
||||
console.error('other methods target public paths (see src/services/supply-chain/index.ts).\n');
|
||||
console.error('If a single class needs both gated and ungated calls, split into two');
|
||||
console.error('instances — one with premiumFetch (used for premium methods) and one with');
|
||||
console.error('globalThis.fetch (used for public methods only). See src/services/trade/');
|
||||
console.error('index.ts for the publicClient + premiumClient pattern.\n');
|
||||
console.error('Reference: HIGH(new) #1 in #3242 review — SupplyChainServiceClient was');
|
||||
console.error('constructed with globalThis.fetch and pro users saw silent empty country-');
|
||||
console.error('products + multi-sector-cost-shock panels until commit 01518c3c.');
|
||||
process.exit(1);
|
||||
}
|
||||
|
||||
console.log(
|
||||
`\u2713 premium-fetch parity clean: ${clientClassMap.size} ServiceClient classes scanned, ${premiumPaths.size} premium paths checked, ${files.length} src/ files analyzed.`,
|
||||
);
|
||||
}
|
||||
|
||||
main().catch((err) => {
|
||||
console.error(err);
|
||||
process.exit(1);
|
||||
});
|
||||
@@ -1,4 +1,4 @@
|
||||
#!/usr/bin/env node
|
||||
#!/usr/bin/env -S npx tsx
|
||||
/**
|
||||
* Validates every key in ENDPOINT_RATE_POLICIES (server/_shared/rate-limit.ts)
|
||||
* is a real gateway route by checking the OpenAPI specs generated from protos.
|
||||
@@ -8,50 +8,54 @@
|
||||
* `/api/sanctions/v1/lookup-sanction-entity`, so the 30/min limit never
|
||||
* applied and the endpoint fell through to the 600/min global limiter).
|
||||
*
|
||||
* Runs in the same pre-push + CI context as lint:api-contract.
|
||||
* Runs in the same pre-push + CI context as lint:api-contract. Invoked via
|
||||
* `tsx` so it can import the policy object straight from the TS source
|
||||
* (#3278) — the previous regex-parse implementation would silently break if
|
||||
* the source object literal was reformatted.
|
||||
*/
|
||||
import { readFileSync, readdirSync } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { pathToFileURL } from 'node:url';
|
||||
import { parse as parseYaml } from 'yaml';
|
||||
|
||||
const ROOT = new URL('..', import.meta.url).pathname;
|
||||
const OPENAPI_DIR = join(ROOT, 'docs/api');
|
||||
const RATE_LIMIT_SRC = join(ROOT, 'server/_shared/rate-limit.ts');
|
||||
|
||||
function extractPolicyKeys() {
|
||||
const src = readFileSync(RATE_LIMIT_SRC, 'utf8');
|
||||
const match = src.match(/ENDPOINT_RATE_POLICIES:\s*Record<[^>]+>\s*=\s*\{([\s\S]*?)\n\};/);
|
||||
if (!match) {
|
||||
throw new Error('Could not locate ENDPOINT_RATE_POLICIES in rate-limit.ts');
|
||||
async function extractPolicyKeys() {
|
||||
// Dynamic import via the file URL — works under tsx (the shebang) which
|
||||
// transparently transpiles TS. Importing the live object means any reformat
|
||||
// of the source literal can never desync the lint from the runtime.
|
||||
const mod = await import(pathToFileURL(RATE_LIMIT_SRC).href);
|
||||
if (!mod.ENDPOINT_RATE_POLICIES || typeof mod.ENDPOINT_RATE_POLICIES !== 'object') {
|
||||
throw new Error(
|
||||
`${RATE_LIMIT_SRC} no longer exports ENDPOINT_RATE_POLICIES — the lint relies on it (#3278).`,
|
||||
);
|
||||
}
|
||||
const block = match[1];
|
||||
const keys = [];
|
||||
// Match quoted keys: '/api/...' or "/api/..."
|
||||
const keyRe = /['"](\/api\/[^'"]+)['"]\s*:/g;
|
||||
let m;
|
||||
while ((m = keyRe.exec(block)) !== null) {
|
||||
keys.push(m[1]);
|
||||
}
|
||||
return keys;
|
||||
return Object.keys(mod.ENDPOINT_RATE_POLICIES);
|
||||
}
|
||||
|
||||
function extractRoutesFromOpenApi() {
|
||||
// Parse the OpenAPI YAML rather than regex-scrape for top-level `paths:`
|
||||
// keys — the earlier `/^\s{4}(\/api\/[^\s:]+):/gm` hard-coded 4-space
|
||||
// indent, so any YAML formatter change (2-space indent, flow style, line
|
||||
// folding) would silently drop routes and let policy-drift slip through
|
||||
// (#3287 greptile nit 3).
|
||||
const routes = new Set();
|
||||
const files = readdirSync(OPENAPI_DIR).filter((f) => f.endsWith('.openapi.yaml'));
|
||||
for (const file of files) {
|
||||
const yaml = readFileSync(join(OPENAPI_DIR, file), 'utf8');
|
||||
// OpenAPI paths section — each route is a top-level key under `paths:`
|
||||
// indented 4 spaces. Strip trailing colon.
|
||||
const pathRe = /^\s{4}(\/api\/[^\s:]+):/gm;
|
||||
let m;
|
||||
while ((m = pathRe.exec(yaml)) !== null) {
|
||||
routes.add(m[1]);
|
||||
const doc = parseYaml(readFileSync(join(OPENAPI_DIR, file), 'utf8'));
|
||||
const paths = doc?.paths;
|
||||
if (!paths || typeof paths !== 'object') continue;
|
||||
for (const route of Object.keys(paths)) {
|
||||
if (route.startsWith('/api/')) routes.add(route);
|
||||
}
|
||||
}
|
||||
return routes;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const keys = extractPolicyKeys();
|
||||
async function main() {
|
||||
const keys = await extractPolicyKeys();
|
||||
const routes = extractRoutesFromOpenApi();
|
||||
const missing = keys.filter((k) => !routes.has(k));
|
||||
|
||||
@@ -73,4 +77,7 @@ function main() {
|
||||
console.log(`✓ rate-limit policies clean: ${keys.length} policies validated against ${routes.size} gateway routes.`);
|
||||
}
|
||||
|
||||
main();
|
||||
main().catch((err) => {
|
||||
console.error(err);
|
||||
process.exit(1);
|
||||
});
|
||||
|
||||
@@ -77,7 +77,11 @@ interface EndpointRatePolicy {
|
||||
window: Duration;
|
||||
}
|
||||
|
||||
const ENDPOINT_RATE_POLICIES: Record<string, EndpointRatePolicy> = {
|
||||
// Exported so scripts/enforce-rate-limit-policies.mjs can import it directly
|
||||
// (#3278) instead of regex-parsing this file. Internal callers should keep
|
||||
// using checkEndpointRateLimit / hasEndpointRatePolicy below — the export is
|
||||
// for tooling, not new runtime callers.
|
||||
export const ENDPOINT_RATE_POLICIES: Record<string, EndpointRatePolicy> = {
|
||||
'/api/news/v1/summarize-article-cache': { limit: 3000, window: '60 s' },
|
||||
'/api/intelligence/v1/classify-event': { limit: 600, window: '60 s' },
|
||||
// Legacy /api/sanctions-entity-search rate limit was 30/min per IP. Preserve
|
||||
|
||||
@@ -147,15 +147,40 @@ function staleToProto(f: StaleFlight): ListMilitaryFlightsResponse['flights'][nu
|
||||
};
|
||||
}
|
||||
|
||||
// Negative cache for the stale Redis read — mirrors the legacy
|
||||
// /api/military-flights handler's NEG_TTL=30_000ms. When the live fetch fails
|
||||
// AND the stale key is also empty/unparseable, suppress further Redis reads
|
||||
// of REDIS_STALE_KEY for STALE_NEG_TTL_MS so we don't hammer Redis once per
|
||||
// request during sustained relay+seed outages. Per-isolate (Vercel Edge state),
|
||||
// which is fine — each warm isolate gets its own 30s suppression window.
|
||||
const STALE_NEG_TTL_MS = 30_000;
|
||||
let staleNegUntil = 0;
|
||||
|
||||
// Test seam — exposed for unit tests that need to drive the suppression
|
||||
// window without sleeping. Not exported from the module's public API.
|
||||
export function _resetStaleNegativeCacheForTests(): void {
|
||||
staleNegUntil = 0;
|
||||
}
|
||||
|
||||
async function fetchStaleFallback(): Promise<ListMilitaryFlightsResponse['flights'] | null> {
|
||||
const now = Date.now();
|
||||
if (now < staleNegUntil) return null;
|
||||
try {
|
||||
const raw = (await getRawJson(REDIS_STALE_KEY)) as StalePayload | null;
|
||||
if (!raw || !Array.isArray(raw.flights) || raw.flights.length === 0) return null;
|
||||
if (!raw || !Array.isArray(raw.flights) || raw.flights.length === 0) {
|
||||
staleNegUntil = now + STALE_NEG_TTL_MS;
|
||||
return null;
|
||||
}
|
||||
const flights = raw.flights
|
||||
.map(staleToProto)
|
||||
.filter((f): f is NonNullable<typeof f> => f != null);
|
||||
return flights.length > 0 ? flights : null;
|
||||
if (flights.length === 0) {
|
||||
staleNegUntil = now + STALE_NEG_TTL_MS;
|
||||
return null;
|
||||
}
|
||||
return flights;
|
||||
} catch {
|
||||
staleNegUntil = now + STALE_NEG_TTL_MS;
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
@@ -66,12 +66,16 @@ export async function registerWebhook(
|
||||
]);
|
||||
}
|
||||
|
||||
// Proto default int32 is 0 — treat 0 as "unset" to preserve the legacy
|
||||
// default of 50 when the caller omits alertThreshold.
|
||||
const alertThreshold = req.alertThreshold > 0 ? req.alertThreshold : 50;
|
||||
// alert_threshold is `optional int32` (#3242 followup #4) — undefined means
|
||||
// the partner omitted the field, so apply the legacy default of 50. An
|
||||
// explicit 0 is preserved (deliver every alert). The 0..100 range is
|
||||
// normally enforced by buf.validate at the wire layer, but we re-enforce
|
||||
// it here so direct handler calls (internal jobs, test harnesses, future
|
||||
// transports that bypass buf.validate) can't store out-of-range values.
|
||||
const alertThreshold = req.alertThreshold ?? 50;
|
||||
if (alertThreshold < 0 || alertThreshold > 100) {
|
||||
throw new ValidationError([
|
||||
{ field: 'alertThreshold', description: 'alertThreshold must be a number between 0 and 100' },
|
||||
{ field: 'alertThreshold', description: 'alertThreshold must be between 0 and 100' },
|
||||
]);
|
||||
}
|
||||
|
||||
|
||||
@@ -4,8 +4,24 @@ export const WEBHOOK_TTL = 86400 * 30; // 30 days
|
||||
export const VALID_CHOKEPOINT_IDS = new Set(CHOKEPOINT_REGISTRY.map(c => c.id));
|
||||
|
||||
// Private IP ranges + known cloud metadata hostnames blocked at registration.
|
||||
// DNS rebinding is not mitigated here (no DNS resolution in edge runtime); the
|
||||
// delivery worker must re-resolve and re-check before sending.
|
||||
//
|
||||
// DNS rebinding is NOT mitigated by isBlockedCallbackUrl below — the Vercel
|
||||
// Edge runtime can't resolve hostnames before the request goes out. Defense
|
||||
// against a hostname that returns a public IP at registration time and a
|
||||
// private IP later (or different IPs per resolution) MUST happen in the
|
||||
// delivery worker that actually POSTs to the callback URL:
|
||||
//
|
||||
// 1. Re-validate the URL with isBlockedCallbackUrl right before each send.
|
||||
// 2. Resolve the hostname to its current IP via dns.promises.lookup
|
||||
// (Node runtime — Edge can't do this).
|
||||
// 3. Verify the resolved IP is not in PRIVATE_HOSTNAME_PATTERNS or
|
||||
// BLOCKED_METADATA_HOSTNAMES.
|
||||
// 4. Issue the fetch using the resolved IP with the Host header preserved
|
||||
// so TLS still validates against the original hostname.
|
||||
//
|
||||
// As of the #3242 followup audit, no delivery worker for shipping/v2 webhooks
|
||||
// exists in this repo — tracked in issue #3288. Anyone landing delivery code
|
||||
// MUST import the patterns + sets above and apply steps 1–3 before each send.
|
||||
export const PRIVATE_HOSTNAME_PATTERNS = [
|
||||
/^localhost$/i,
|
||||
/^127\.\d+\.\d+\.\d+$/,
|
||||
|
||||
@@ -39,7 +39,7 @@ export interface BypassOption {
|
||||
export interface RegisterWebhookRequest {
|
||||
callbackUrl: string;
|
||||
chokepointIds: string[];
|
||||
alertThreshold: number;
|
||||
alertThreshold?: number;
|
||||
}
|
||||
|
||||
export interface RegisterWebhookResponse {
|
||||
|
||||
@@ -39,7 +39,7 @@ export interface BypassOption {
|
||||
export interface RegisterWebhookRequest {
|
||||
callbackUrl: string;
|
||||
chokepointIds: string[];
|
||||
alertThreshold: number;
|
||||
alertThreshold?: number;
|
||||
}
|
||||
|
||||
export interface RegisterWebhookResponse {
|
||||
|
||||
@@ -9,6 +9,7 @@ import type {
|
||||
} from './types';
|
||||
import { haversineKm } from '@/utils/distance';
|
||||
import { IntelligenceServiceClient } from '@/generated/client/worldmonitor/intelligence/v1/service_client';
|
||||
import { premiumFetch } from '@/services/premium-fetch';
|
||||
import { hasPremiumAccess } from '@/services/panel-gating';
|
||||
|
||||
const LLM_SCORE_THRESHOLD = 60;
|
||||
@@ -30,8 +31,11 @@ export class CorrelationEngine {
|
||||
private llmInFlight = 0;
|
||||
|
||||
constructor() {
|
||||
// Use '' base URL — requests go to current origin, same as other panels
|
||||
this.intelligenceClient = new IntelligenceServiceClient('');
|
||||
// Use '' base URL — requests go to current origin, same as other panels.
|
||||
// premiumFetch — deductSituation is in PREMIUM_RPC_PATHS. globalThis.fetch
|
||||
// (the generated default) would 401 signed-in browser pros so the LLM
|
||||
// assessment never lands. See #3242 review HIGH(new) #1 for the bug class.
|
||||
this.intelligenceClient = new IntelligenceServiceClient('', { fetch: premiumFetch });
|
||||
}
|
||||
|
||||
registerAdapter(adapter: DomainAdapter): void {
|
||||
|
||||
@@ -8,6 +8,7 @@
|
||||
*/
|
||||
|
||||
import { getRpcBaseUrl } from '@/services/rpc-client';
|
||||
import { premiumFetch } from '@/services/premium-fetch';
|
||||
import {
|
||||
EconomicServiceClient,
|
||||
ApiError,
|
||||
@@ -53,7 +54,14 @@ import { toApiUrl } from '@/services/runtime';
|
||||
|
||||
// ---- Client + Circuit Breakers ----
|
||||
|
||||
const client = new EconomicServiceClient(getRpcBaseUrl(), { fetch: (...args) => globalThis.fetch(...args) });
|
||||
// premiumFetch for the whole client: 1 of ~16 methods (getNationalDebt) targets a
|
||||
// PREMIUM_RPC_PATHS path. globalThis.fetch here would 401 signed-in browser pros
|
||||
// on getNationalDebt with no WORLDMONITOR_API_KEY (gateway runs validateApiKey
|
||||
// with forceKey=true on premium paths). premiumFetch no-ops safely when no
|
||||
// credentials are available, so the public methods (FRED, BLS, energy, BIS,
|
||||
// EU, oil) keep working unchanged. See src/services/supply-chain/index.ts for
|
||||
// the same pattern + #3242 review HIGH(new) #1 for the bug class this prevents.
|
||||
const client = new EconomicServiceClient(getRpcBaseUrl(), { fetch: premiumFetch });
|
||||
const WB_BREAKERS_WARN_THRESHOLD = 50;
|
||||
const wbBreakers = new Map<string, ReturnType<typeof createCircuitBreaker<ListWorldBankIndicatorsResponse>>>();
|
||||
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
import { createCircuitBreaker } from '@/utils';
|
||||
import { getRpcBaseUrl } from '@/services/rpc-client';
|
||||
import { premiumFetch } from '@/services/premium-fetch';
|
||||
import { getHydratedData } from '@/services/bootstrap';
|
||||
import {
|
||||
SanctionsServiceClient,
|
||||
@@ -54,7 +55,10 @@ export interface SanctionsPressureResult {
|
||||
entries: SanctionsEntry[];
|
||||
}
|
||||
|
||||
const client = new SanctionsServiceClient(getRpcBaseUrl(), { fetch: (...args) => globalThis.fetch(...args) });
|
||||
// premiumFetch — listSanctionsPressure (the only method called here) is in
|
||||
// PREMIUM_RPC_PATHS. See src/services/supply-chain/index.ts for the pattern
|
||||
// and #3242 review HIGH(new) #1 for the bug class this prevents.
|
||||
const client = new SanctionsServiceClient(getRpcBaseUrl(), { fetch: premiumFetch });
|
||||
const breaker = createCircuitBreaker<SanctionsPressureResult>({
|
||||
name: 'Sanctions Pressure',
|
||||
cacheTtlMs: 30 * 60 * 1000,
|
||||
|
||||
@@ -878,4 +878,74 @@ describe('military flights bbox behavior', { concurrency: 1 }, () => {
|
||||
restoreEnv();
|
||||
}
|
||||
});
|
||||
|
||||
// #3277 — fetchStaleFallback NEG_TTL parity with the legacy
|
||||
// /api/military-flights handler. Without the negative cache, a sustained
|
||||
// relay+seed outage would Redis-hammer the stale key on every request.
|
||||
it('suppresses stale Redis read for 30s after a stale-key miss (NEG_TTL parity)', async () => {
|
||||
const { module, cleanup } = await importListMilitaryFlights();
|
||||
module._resetStaleNegativeCacheForTests();
|
||||
|
||||
const restoreEnv = withEnv({
|
||||
UPSTASH_REDIS_REST_URL: 'https://redis.test',
|
||||
UPSTASH_REDIS_REST_TOKEN: 'token',
|
||||
LOCAL_API_MODE: undefined,
|
||||
WS_RELAY_URL: undefined,
|
||||
VERCEL_ENV: undefined,
|
||||
VERCEL_GIT_COMMIT_SHA: undefined,
|
||||
});
|
||||
const originalFetch = globalThis.fetch;
|
||||
|
||||
const staleGetCalls = [];
|
||||
globalThis.fetch = async (url) => {
|
||||
const raw = String(url);
|
||||
if (raw.includes('/get/')) {
|
||||
if (raw.includes('military%3Aflights%3Astale%3Av1')) {
|
||||
staleGetCalls.push(raw);
|
||||
}
|
||||
// Both keys empty — drives cachedFetchJson to call the fetcher
|
||||
// (which returns null because no relay) and then the handler falls
|
||||
// through to fetchStaleFallback (which returns null because stale
|
||||
// is also empty → arms the negative cache).
|
||||
return jsonResponse({ result: null });
|
||||
}
|
||||
throw new Error(`Unexpected fetch URL: ${raw}`);
|
||||
};
|
||||
|
||||
try {
|
||||
const ctx = { request: new Request('https://wm.test/api/military/v1/list-military-flights') };
|
||||
|
||||
// Call 1 — live empty + stale empty. Stale key MUST be read once,
|
||||
// and the negative cache MUST be armed for the next 30s.
|
||||
const r1 = await module.listMilitaryFlights(ctx, request);
|
||||
assert.deepEqual(r1.flights, [], 'no live, no stale → empty response');
|
||||
assert.equal(staleGetCalls.length, 1, 'first call reads stale key once');
|
||||
|
||||
// Call 2 — within the 30s negative-cache window. Live cache may be
|
||||
// re-checked but the stale key MUST NOT be re-read.
|
||||
staleGetCalls.length = 0;
|
||||
const r2 = await module.listMilitaryFlights(ctx, request);
|
||||
assert.deepEqual(r2.flights, [], 'still empty during negative-cache window');
|
||||
assert.equal(
|
||||
staleGetCalls.length,
|
||||
0,
|
||||
'second call within NEG_TTL window must not re-read stale key',
|
||||
);
|
||||
|
||||
// Reset the negative cache (simulates wall-clock advance past 30s) →
|
||||
// stale read should resume.
|
||||
module._resetStaleNegativeCacheForTests();
|
||||
const r3 = await module.listMilitaryFlights(ctx, request);
|
||||
assert.deepEqual(r3.flights, []);
|
||||
assert.equal(
|
||||
staleGetCalls.length,
|
||||
1,
|
||||
'after negative-cache reset, stale key is re-read',
|
||||
);
|
||||
} finally {
|
||||
cleanup();
|
||||
globalThis.fetch = originalFetch;
|
||||
restoreEnv();
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
@@ -207,12 +207,27 @@ describe('ShippingV2Service handlers', () => {
|
||||
);
|
||||
});
|
||||
|
||||
// alert_threshold 0..100 range is enforced primarily by buf.validate at
|
||||
// the wire layer. The handler re-enforces it so direct invocations
|
||||
// (internal jobs, test harnesses, future transports) can't store out-of-
|
||||
// range values — cheap invariant-at-the-boundary (#3287 review nit 1).
|
||||
it('rejects alertThreshold > 100 with ValidationError', async () => {
|
||||
await assert.rejects(
|
||||
() => registerWebhook(proCtx(), {
|
||||
callbackUrl: 'https://hooks.example.com/wm',
|
||||
chokepointIds: [],
|
||||
alertThreshold: 150,
|
||||
alertThreshold: 9999,
|
||||
}),
|
||||
(err) => err instanceof ValidationError && err.violations[0].field === 'alertThreshold',
|
||||
);
|
||||
});
|
||||
|
||||
it('rejects alertThreshold < 0 with ValidationError', async () => {
|
||||
await assert.rejects(
|
||||
() => registerWebhook(proCtx(), {
|
||||
callbackUrl: 'https://hooks.example.com/wm',
|
||||
chokepointIds: [],
|
||||
alertThreshold: -1,
|
||||
}),
|
||||
(err) => err instanceof ValidationError && err.violations[0].field === 'alertThreshold',
|
||||
);
|
||||
@@ -246,7 +261,22 @@ describe('ShippingV2Service handlers', () => {
|
||||
assert.equal(pipeline[2][2], String(86400 * 30));
|
||||
});
|
||||
|
||||
it('alertThreshold 0 (proto default) coerces to legacy default 50', async () => {
|
||||
it('alertThreshold omitted (undefined) applies the legacy default of 50', async () => {
|
||||
const calls = stubRedisOk();
|
||||
await registerWebhook(proCtx(), {
|
||||
callbackUrl: 'https://hooks.example.com/wm',
|
||||
chokepointIds: [],
|
||||
// alertThreshold omitted — proto3 `optional int32` arrives as undefined
|
||||
});
|
||||
const record = JSON.parse(calls[0][0][2]);
|
||||
assert.equal(record.alertThreshold, 50);
|
||||
});
|
||||
|
||||
it('alertThreshold explicit 0 is preserved (deliver every alert)', async () => {
|
||||
// #3242 followup #4 — proto3 `optional` lets the handler distinguish
|
||||
// "partner explicitly sent 0" from "partner omitted the field". The
|
||||
// pre-fix handler coerced both to 50, silently dropping the partner's
|
||||
// intent to receive every disruption.
|
||||
const calls = stubRedisOk();
|
||||
await registerWebhook(proCtx(), {
|
||||
callbackUrl: 'https://hooks.example.com/wm',
|
||||
@@ -254,7 +284,7 @@ describe('ShippingV2Service handlers', () => {
|
||||
alertThreshold: 0,
|
||||
});
|
||||
const record = JSON.parse(calls[0][0][2]);
|
||||
assert.equal(record.alertThreshold, 50);
|
||||
assert.equal(record.alertThreshold, 0);
|
||||
});
|
||||
|
||||
it('empty chokepointIds subscribes to the full CHOKEPOINT_REGISTRY', async () => {
|
||||
|
||||
Reference in New Issue
Block a user