* fix(digest): never skip AI summary when userPreferences are missing
Users who enabled the AI executive summary toggle on their notification
rule still received digest emails without the summary. The Railway log
pinpointed it:
[digest] No preferences for user_... skipping AI summary
[digest] Email delivered to ...
Root cause chain:
convex/http.ts:591 /relay/user-preferences returns literal
null when no userPreferences row exists
for (userId, variant).
scripts/lib/user-context.cjs fetchUserPreferences forwards that as
{ data: null, error: false }.
scripts/seed-digest-notifications.mjs:458
generateAISummary bails with return null.
The AI-summary toggle lives on the alertRules table. userPreferences is
a SEPARATE table (the SPA app-settings blob: watchlist, airports,
panels). A user can have an alertRule (with aiDigestEnabled: true)
without having ever saved userPreferences, or only under a different
variant. Missing prefs must NOT silently disable the feature the user
explicitly enabled. The correct behavior is to degrade to a
non-personalized summary.
Fix: remove the early return in generateAISummary. Call
extractUserContext(null), which already returns a safe empty context,
and formatUserProfile(ctx, 'full') returns "Variant: full" alone. The
LLM then generates a generic daily brief instead of nothing. An info
log still reports the missing-prefs case for observability.
Regression coverage: tests/user-context.test.mjs (new, 10 cases) locks
in that extractUserContext(null|undefined|{}|"") returns the empty
shape and formatUserProfile(emptyCtx, variant) returns exactly
"Variant: {variant}". Any future refactor that reintroduces the
null-bail will fail the tests.
Note: the same log also shows the rule fired at 13:01 Dubai instead
of 8 AM / 8 PM. That is a separate issue in isDue or rule-save flow
and needs more log data to diagnose; not included here.
* fix(digest): distinguish transient prefs fetch failure from missing row
Addresses Greptile P2 review feedback on PR #2939.
fetchUserPreferences returns { data, error } where:
error: true = transient fetch failure (network, non-OK HTTP, env missing)
error: false = the (userId, variant) row genuinely does not exist
The previous log treated both cases identically as "No stored preferences",
which was misleading when the real cause was an unreachable Convex endpoint.
Behavior is unchanged (both still degrade to a non-personalized summary),
only the log line differentiates them so transient fetch failures are
visible in observability.
* feat(sentiment): add AAII investor sentiment survey
Weekly bull/bear/neutral sentiment from AAII (1987-present). Shows
current reading, bull-bear spread, and 52-week historical chart.
Seeder fetches from AAII CSV, stores last 52 weeks in Redis.
* fix(aaii): wire panel loading + mark fallback data explicitly
* fix(aaii): keep panel live across refreshes + surface in health monitoring
- fetchData now falls back to /api/bootstrap?keys=aaiiSentiment on
refresh (getHydratedData is one-shot and returns undefined after
the first read, causing a permanent spinner on hourly refresh)
- Shows an error state with auto-retry when both hydrated and
bootstrap-fetch miss, matching the WsbTickerScannerPanel pattern
- Registered aaiiSentiment in api/health.js BOOTSTRAP_KEYS and
api/seed-health.js SEED_DOMAINS so rollout failures and
fallback-only operation are observable in the monitoring dashboards
* fix(sentiment): handle BIFF8 SST trailing bytes and use UTC for AAII Thursday calc
Two P2 greptile fixes from PR #2930 review:
1. BIFF8 SST parser was reading the rich-text run count (cRun, flags & 0x08)
and extended-string size (cbExtRst, flags & 0x04) to advance past those
header fields, but never skipped the trailing bytes AFTER the char data:
4 * cRun formatting-run bytes and cbExtRst ext-rst bytes. If any string
before the column header was rich-text formatted, every subsequent SST
entry parsed from the wrong offset, silently breaking XLS extraction and
falling back to HTML scraping.
2. parseHtmlSentiment() computed last-Thursday via today.getDay() +
setDate(today.getDate() - daysToThursday), both local-TZ-dependent. On
Railway (non-UTC TZ) the inferred Thursday could drift by a day, causing
the HTML-derived row to mismatch the XLS historical rows. Switched to
getUTCDay() + Date.UTC() for TZ-stable arithmetic.
* feat(sectors): add P/E valuation benchmarking to sector heatmap
Trailing/forward P/E, beta, and returns for 12 sector ETFs from Yahoo
Finance. Horizontal bar chart color-coded by valuation level plus
sortable table. Extends existing sector data pipeline.
* fix(sectors): clear stale valuations on empty refresh + document cache behavior
* fix(sectors): force valuation rollout for cached + breaker-persisted bootstraps
- Bumped market:sectors bootstrap key v1 -> v2 so stale 24h slow-tier
payloads without the new valuations field are invisible to returning
users on next page load
- Versioned the fetchSectors circuit-breaker (name -> "Sector Summary v2")
so old localStorage/IndexedDB entries predating this PR cannot be
returned as stale via the SWR path
- shouldCache now requires the valuations field to be present on the
cached response, not just a non-empty sectors array
- loadMarkets no longer clears the valuations tab when a hydrated or
fresh payload lacks the field; prior render is left intact, matching
the finding's requirement
- Defensive check: hydrated payloads without valuations fall through to
a live fetch instead of rendering an empty valuations tab
* fix(stocks): correct beta3Year source and null YTD color in sector P/E view
- scripts/ais-relay.cjs: beta3Year lives on defaultKeyStatistics (ks),
not summaryDetail (sd); the previous fallback was a silent no-op.
- src/components/MarketPanel.ts: null ytdReturn now renders with
var(--text-dim) instead of var(--red); the '--' placeholder no
longer looks like a loss.
Addresses greptile review on PR #2929.
* feat(stocks): add insider transaction tracking to stock analysis panel
Shows 6-month insider buy/sell activity from Finnhub: total buys,
sells, net value, and recent named-exec transactions. Gracefully
skips when FINNHUB_API_KEY is unavailable.
* fix: add cache tier entry for get-insider-transactions route
* fix(stocks): add insider RPC to premium paths + fix empty/stale states
* fix(stocks): add insider RPC to premium paths + fix empty/stale states
- Add /api/market/v1/get-insider-transactions to PREMIUM_RPC_PATHS
- Return unavailable:false with empty transactions when Finnhub has no data
(panel shows "No insider transactions" instead of "unavailable")
- Mark stale insider data on refresh failures to avoid showing outdated info
- Update test to match new empty-data behavior
* fix(stocks): unblock stock-analysis render and surface exercise-only insider activity
- loadStockAnalysis no longer awaits loadInsiderDataForPanel before
panel.renderAnalyses. The insider fetch now fires in parallel after
the primary render at both the cached-snapshot and live-fetch call
sites. When insider data arrives, loadInsiderDataForPanel re-renders
the panel so the section fills in asynchronously without holding up
the analyst report on a secondary Finnhub RPC.
- Add transaction code 'M' (exercise / conversion of derivative) to
the allowed set in get-insider-transactions so symbols whose only
recent Form 4 activity is option/RSU exercises no longer appear as
"No insider transactions in the last 6 months". Exercises do not
contribute to buys/sells dollar totals because transactionPrice is
the strike price, not a market transaction.
- Panel table now uses a neutral (dim) color for non-buy/non-sell
rows (M rows) instead of the buy/sell green/red binary.
- Tests cover: exercise-only activity producing non-empty transactions
with zero buys/sells, and blended P/S/M activity preserving all
three rows.
* fix(stocks): prevent cached insider fetch from clobbering live render
- Cached-path insider enrichment only runs when no live fetch is coming
- Added generation counter to guard against concurrent loadStockAnalysis calls
- Stale insider fetches now no-op instead of reverting panel state
* fix(stocks): hide transient insider-unavailable flash and zero out strike-derived values
- renderInsiderSection returns empty string when insider data is not yet
fetched, so the transient "Insider data unavailable" card no longer
flashes on initial render before the RPC completes
- Exercise rows (code M) now carry value: 0 on the server and render a
dash placeholder in the Value cell, matching how the buy/sell totals
already exclude strike-derived dollar amounts
* fix(stocks): exclude non-market Form 4 codes (A/D/F) from insider buy/sell totals
Form 4 codes A (grant/award), D (disposition to issuer), and F (tax/exercise
payment) are not open-market trades and should not drive insider conviction
totals. Only P (open-market purchase) and S (open-market sale) now feed the
buy/sell aggregates. A/D/F rows are still surfaced in the transaction list
alongside M (exercise) with value zeroed out so the panel does not look empty.
* feat(stocks): add dividend growth analysis to stock analysis panel
Shows yield, 5Y CAGR, frequency (quarterly/monthly/annual), payout
ratio, and ex-dividend date. Hidden for non-dividend stocks. Data
from Yahoo Finance dividend history.
* fix(stocks): bump cache key + fix partial-year CAGR + remove misleading avg yield
* fix(stocks): hydrate payout ratio, drop dead five-year yield, currency-aware dividend rate
- Add fetchPayoutRatio helper that calls Yahoo quoteSummary's summaryDetail
module in parallel with the dividend chart fetch and returns the raw 0-1
payout ratio (or undefined when missing/non-positive). The chart endpoint
alone never returns payoutRatio, which is why it was hardcoded to 0.
- Make payout_ratio optional in proto and DividendProfile so a missing value
is undefined instead of 0; remove five_year_avg_dividend_yield entirely
(proto reserved 51) since it was always returned as 0 and never wired up.
- StockAnalysisPanel.renderDividendProfile now omits the Payout Ratio cell
unless the value is present and > 0, formats it as (raw * 100).toFixed(1)%,
and renders the dividend rate via Intl.NumberFormat with item.currency so
EUR/GBP/JPY tickers no longer get a hardcoded "$" prefix.
- Bump live cache key v2 -> v3 so any cached snapshots persisted with the
old shape are refetched once.
- Tests cover: payoutRatio populated from summaryDetail, payoutRatio
undefined when summaryDetail returns 500 or raw=0, dividend response
shape no longer contains fiveYearAvgDividendYield.
* fix(stocks): bump persisted history store to v3 to rotate pre-PR snapshots
Live analyze-stock cache was already bumped to v3, but the persisted
history store (premium-stock-store.ts) still used v2 keys for index/item
lookups. Pre-PR snapshots without the new dividend fields could pass the
age-only freshness check and suppress a live refetch, leaving the new
dividend section missing for up to 15 minutes.
Bumping the persisted store keys to v3 makes old snapshots invisible.
The data loader sees an empty history, triggers a live refetch, and
repopulates under the new v3 keys. Old v2 keys expire via TTL.
* fix(stocks): compute dividend CAGR correctly for quarterly/semiannual/annual payers
Previously computeDividendCagr() required at least 10 distinct dividend
months for a year to count as "full", which excluded every non-monthly
dividend payer (quarterly = 4 months, semiannual = 2, annual = 1).
CAGR therefore collapsed to 0/N/A for most ordinary dividend stocks.
The new check uses calendar position: any year strictly earlier than the
current calendar year is treated as complete, and the current in-progress
year is excluded to avoid penalizing stocks whose next payment has not
yet landed.
* test(stocks): pass analystData to buildAnalysisResponse after rebase onto #2926
* feat(stocks): add analyst consensus + price targets to stock analysis panel
Shows recommendation trend (strongBuy/buy/hold/sell), price target range
(high/low/median vs current), and recent upgrade/downgrade actions with
firm names. Data from Yahoo Finance quoteSummary.
* chore: regenerate proto types and OpenAPI docs
* fix(stocks): fallback median to mean + use stock currency for price targets
* fix(stocks): drop fake $0 price targets and force refetch for pre-rollout snapshots
- Make PriceTarget high/low/mean/median/current optional in proto so partial
Yahoo financialData payloads stop materializing as $0.00 cells in the panel.
- fetchYahooAnalystData now passes undefined (via optionalPositive) when a
field is missing or non-positive, instead of coercing to 0.
- StockAnalysisPanel.renderPriceTarget skips Low/High cells entirely when the
upstream value is missing and falls back to a Median + Analysts view.
- Add field-presence freshness check in stock-analysis-history: snapshots
written before the analyst-revisions rollout (no analystConsensus and no
priceTarget) are now classified as stale even when their generatedAt is
inside the freshness window, so the data loader forces a live refetch.
- Tests cover undefined targets path, missing financialData path, and the
three field-presence freshness branches.
* fix(stocks): preserve fresh snapshots on partial refetch + accept median-only targets
- loadStockAnalysis now merges still-fresh cached symbols with refetched
live results so a partial refetch does not shrink the rendered watchlist
- renderAnalystConsensus accepts median-only price targets (not just mean)
* feat(deep-dive): alternative supplier risk assessment with chokepoint routing
Add client-side route risk computation for product importers. Each exporter
in the Product Imports table now shows a risk badge (Safe/At Risk/Critical)
based on whether its trade routes transit disrupted chokepoints. Recommendations
section suggests safer alternatives when critical chokepoints are detected.
* fix(test): update deep-dive panel harness stubs for supplier risk imports
Add fetchChokepointStatus, TRADE_ROUTES, chokepoint-registry, and
supplier-route-risk stubs so the esbuild-based resilience test passes
with the new supplier risk assessment feature.
* fix(deep-dive): address code review findings for supplier risk
- Remove sort in computeAlternativeSuppliers to preserve trade-share order
- Replace silent .catch(() => {}) with console.warn for debuggability
- Remove double escapeHtml on textContent assignments (textContent auto-escapes)
- Guard reduce on empty transitChokepoints array in buildRecommendation and panel
- Replace double type assertion with proper CountryPortClustersJson interface
- Add cdp-recommendation-critical CSS class, use it for critical vs at_risk
- Use explicit ExporterRow type alias for type-safe fallback path (no null casts)
* fix(deep-dive): distinguish unknown from safe in supplier route risk
Exporters with no cluster entry or no overlapping routes now get
riskLevel 'unknown' instead of 'safe'. This prevents unmodeled
exporters from being recommended as safe alternatives. Adds gray
'Unknown' badge in the UI and skips unknown exporters in the
recommendations section.
* fix(deep-dive): add stale-guard to chokepoint fetch in product detail
Capture the current country code before fetchChokepointStatus() and
bail in the callback if the user switched countries while the request
was in-flight. Prevents enrichment from running on detached DOM nodes.
* fix(trade): fetch Budget Lab tariff from GitHub embed + new pattern
The Budget Lab page is now a Next.js SPA (no static HTML content).
The report is embedded from GitHub: Budget-Lab-Yale/tariff-impact-
tracker/website/html/tariff_impacts_report_drupal.html
Changes:
- Fetch from GitHub raw URL first (stable), fall back to original
- Added "stood at X%" regex pattern which matches the current rate
(11.1% post-SCOTUS) instead of the older "reaching X%" (10.6%)
* fix(trade): replace Budget Lab HTML scraper with FRED API
Budget Lab page is a Next.js SPA, scraping kept breaking.
Now computes US effective tariff rate from official FRED data:
- B235RC1Q027SBEA (customs duties, quarterly SAAR)
- IEAMGSN (goods imports, quarterly SAAR)
- Rate = customs / imports × 100
Uses existing fredFetchJson() infrastructure. No scraping,
no fragile regex, no SPA bypass needed. Pure API.
* fix(trade): build full FRED API URLs for tariff rate series
fredFetchJson() expects a full URL, not a series ID.
Built proper FRED API URLs with series_id, api_key, and params.
Also extract .observations from FRED response structure.
* fix(trade): use matching FRED import series for tariff rate
IEAMGSN (Millions, NSA) was incompatible with B235RC1Q027SBEA
(Billions, SAAR), producing ~0.03% instead of ~11%.
Switched to A255RC1Q027SBEA (Imports of goods, Billions, Quarterly,
SAAR) which matches the customs duties series exactly.
Verified: 364.3B / 3217.4B × 100 = 11.3% (Q4 2025).
* test(trade): update tariff test for FRED-based effective rate
* fix(trade): pass _proxyAuth to FRED tariff rate calls
* feat(seed): Comtrade bilateral HS4 seeder for 197 countries x 20 products
Add seed-comtrade-bilateral-hs4.mjs that fetches bilateral import data
from UN Comtrade public API at HS4 product level. For each of 197
countries, fetches top 5 exporters per product across 20 strategic HS4
codes (energy, semiconductors, vehicles, pharma, food, etc.).
Includes:
- Rate-limited fetching (3.5s between requests, 60s retry on 429)
- Redis pipeline writes with 72h TTL
- Lock/TTL-extension patterns matching gold standard
- New Vercel edge function api/supply-chain/v1/country-products.ts
(PRO-gated, reads from Upstash Redis)
- fetchCountryProducts() service function with premiumFetch auth
* feat(deep-dive): product imports section with supplier concentration data
Add a PRO-gated "Product Imports" card to CountryDeepDivePanel that
shows top imported products (HS4) with supplier breakdown. Includes a
searchable product selector dropdown, per-product exporter table with
share bars, and value formatting.
Wired into fetchProSections in country-intel.ts so data loads lazily
when a PRO user opens a country deep dive panel.
* feat(seed): use authenticated Comtrade API with key rotation
Switch bilateral HS4 seeder from public preview endpoint to the
authenticated data endpoint (/data/v1/get/) when COMTRADE_API_KEYS
env var is set. Rotates between comma-separated keys on each request
for higher throughput (~500 req/hour per key). Reduces inter-request
delay from 3.5s to 1.5s in authenticated mode. Falls back to the
public preview endpoint when no keys are configured.
* fix(supply-chain): private cache on PRO endpoint, skip writes on fetch failure
Two review fixes for PR #2921:
1. country-products.ts: Change Cache-Control from public to private
with Vary: Authorization, Cookie to prevent CDN/CF from serving
PRO data to non-PRO callers. Empty-data path uses no-store.
2. seed-comtrade-bilateral-hs4.mjs: On per-country fetch failure,
skip the Redis SET entirely so stale-but-valid data is preserved.
Previously the catch block swallowed the error and the code below
still wrote products:[] to Redis, erasing last-known-good data.
* test(supply-chain): add bilateral HS4 seeder and product imports tests
Static analysis tests covering:
- Edge endpoint (country-products.ts): edge config, method guard, iso2 validation,
PRO gating via isCallerPremium, Cache-Control/Vary headers, Upstash reads
- Seeder (seed-comtrade-bilateral-hs4.mjs): distributed lock, isMain guard, key
rotation, TTL, 20 HS4 codes, no-write-on-failure, 429 retry, seed-meta
- Service (supply-chain/index.ts): type exports, premiumFetch usage, graceful fallback
- Panel (CountryDeepDivePanel.ts): updateProductImports, search/filter, PRO gate,
textContent (no innerHTML), resetPanelContent cleanup, sectionCard usage
* fix(supply-chain): guard empty product writes, log retry failures, align interface
1. Skip Redis SET when groupByProduct returns empty (prevents silent
API failures like HTTP 500 from overwriting last-known-good data).
2. Log HTTP status on 429 retry failure for observability.
3. Add partnerCode to ProductExporter interface to match Redis shape.
* feat(wsb): add Reddit/WSB ticker scanner seeder + bootstrap registration
Seeder in ais-relay.cjs fetches from r/wallstreetbets, r/stocks,
r/investing every 10min. Extracts ticker mentions, validates against
known ticker set, aggregates by frequency and engagement, writes top 50
to intelligence:wsb-tickers:v1.
4-file bootstrap registration: cache-keys.ts, bootstrap.js, health.js
with SEED_META maxStaleMin=30.
* fix(wsb): remove duplicate CEO + fix avgUpvoteRatio divisor
* fix(wsb): require ticker validation set + condition seed-meta on write + add seed-health
1. Skip seed when ticker validation set is empty (cold start/bootstrap miss)
2. Only write seed-meta after successful canonical write
3. Register in api/seed-health.js for dedicated monitoring
* fix(wsb): case-insensitive $ticker matching + BRK.B dotted symbol support
* fix(wsb): split $-prefixed vs bare ticker extraction + BRK.B→BRK-B normalization
1. $-prefixed tickers ($nvda, $BRK.B) skip whitelist validation (strong
signal) — catches GME, AMC, PLTR etc. not in the narrow market watchlist
2. Bare uppercase tokens still validated against known set (high false-positive)
3. BRK.B normalized to BRK-B before validation (dot→dash)
4. Empty known set no longer skips seed — $-prefixed tickers still extracted
* fix(wsb): skip bare-uppercase branch entirely when ticker set unavailable
* feat(deep-dive): clickable sector rows with route visualization and bypass corridors
- Trade Exposure sector rows expand on click to show route detail
- Route path: origin > chokepoints > destination from trade-routes config
- Top 3 bypass corridors load lazily on expand (PRO-gated)
- Map highlights selected route (bright arc, others dim to alpha 40)
- Map zooms to fit highlighted route segments
- Auto-minimizes panel if maximized (so map is visible)
- Click again to collapse and restore map
* fix(deep-dive): show all matching routes in detail, fire gate hit on click not render
- buildRouteDetail now iterates all matching routes for a chokepoint,
rendering each as a labeled path (route name + waypoint chain)
- Replaced single matchingRoutes[0] path with full list under a
"Routes via <chokepoint>:" heading
- Moved trackGateHit('sector-bypass-corridors') from render-time to
a { once: true } click handler on the PRO gate placeholder
* fix(deep-dive): clear route highlight before switching sectors
Move clearHighlightedRoute() to the top of handleSectorRowClick()
so it runs unconditionally before any branching. Previously it only
fired when collapsing the current selection, leaving a stale highlight
when switching to a new sector with no matching routes.
* test(deep-dive): add sector route explorer integration tests
Static analysis tests verifying the route highlighting pipeline across
DeckGLMap, MapContainer, and CountryDeepDivePanel. Covers method existence,
dispatch wiring, cleanup in reset, XSS sanitization, and data consistency.
* fix(tests): add missing stubs for sector route explorer imports
The country-deep-dive-panel harness was missing stubs for escapeHtml,
createCircuitBreaker, loadFromStorage, saveToStorage, and new modules
added by sector route explorer (trade-routes, geo, analytics, supply-chain).
* fix(deep-dive): increase port limit to 25 + scrollable table + larger donut
Maritime: raised handler port limit from 5 to 25 (PortWatch seeds up
to 50 per country). Added scroll wrapper (max-height 260px) so the
table doesn't overflow the card.
Energy donut: increased from 90px to 120px (r=46, stroke=18) for
better readability with 8 segments.
* fix(data): correct US UN M49 code from 842 to 840
842 is US Virgin Islands. 840 is the United States. This caused
tariff trends, comtrade flows, and all UN-code-based RPCs to query
the wrong key (trade:tariffs:v1:842:all:10 instead of :840:).
* fix: rename top5→topPorts, sticky table header, fix stale comment
Greptile review: stale variable name, missing sticky header for
25-row scroll container, stale test comment.
* feat(deep-dive): add 6 PRO-only intelligence sections to country brief
Adds National Debt, Sanctions Pressure, Trade Flows (Comtrade),
Tariff Trends, Chokepoint Exposure, and Cost Shock Analysis to the
country deep dive panel. All sections are PRO-gated: free users see
a locked placeholder, PRO users get real-time data.
Data sources: IMF debt data, OFAC sanctions, UN Comtrade, WTO tariffs,
chokepoint vulnerability index, energy shock modeling.
* fix(pro): backend gating for debt/sanctions/trade + canonical ISO maps
P1 fixes from review:
1. Added isCallerPremium() to 4 ungated handlers (get-national-debt,
list-sanctions-pressure, list-comtrade-flows, get-tariff-trends).
Free users now get empty responses, matching supply-chain pattern.
2. Replaced partial hardcoded ISO2-to-ISO3 (80 entries) and ISO2-to-UN
(50 entries) maps with canonical data (239 entries each) in new
src/utils/country-codes.ts. Fixes silent data gaps for countries
missing from the old partial maps.
* fix(pro): register premium paths + use full sanctions data via country-risk
P1 fixes from review round 2:
1. Added 4 newly gated endpoints to PREMIUM_RPC_PATHS so web client
injects auth headers (debt, sanctions, comtrade, tariffs).
2. Replaced truncated sanctions lookup (top-12 countries array) with
getCountryRisk() RPC which reads the full sanctions:country-counts:v1
key covering ALL countries. No more silent "no data" for countries
outside the top-12.
* feat(resilience): add rankStable flag to ranking items
Countries with score interval width <= 8 (p95-p05) are flagged as
rankStable=true, indicating robust ranking under weight perturbation.
Read from batch-computed intervals in Redis.
* fix(resilience): guard inverted intervals + scope fetch to scored countries
1. isRankStable rejects negative width (malformed p05 > p95)
2. fetchIntervals scoped to cachedScores.keys() instead of all countries
* fix(resilience): raw key read for intervals + bump ranking cache to v8
* fix(resilience): remove duplicate ScoreInterval interface after rebase
ScoreInterval is now generated in service_server.ts (from PR #2877).
Remove the local duplicate and re-export the generated type.
Merges the Monte Carlo interval computation into the scores seed script.
After warming scores and reading cached domain data, computes p05/p95
per country and writes resilience:intervals:v1:{ISO2} keys to Redis.
No separate Railway service needed (avoids 100-service limit).
* feat(resilience): add score confidence intervals via batch Monte Carlo
Weekly cron perturbs domain weights ±10% across 100 draws per country,
stores p05/p95 in Redis. Score handler reads intervals and includes
them in the API response as ScoreInterval { p05, p95 }.
Proto field 14 (score_interval) added to GetResilienceScoreResponse.
* chore: regenerate proto types and OpenAPI docs for ScoreInterval
* fix(resilience): add seed-meta + lock + fix interval cache + percentile formula
1. Write seed-meta:resilience:intervals for health monitoring
2. Add distributed lock to prevent concurrent cron overlap
3. Move scoreInterval read outside 6h score cache boundary
4. Fix percentile index from floor to ceil-1 (nearest-rank)
* fix(health): add resilience:intervals to health + seed-health registries
* fix(seed): skip seed-meta on no-op runs + register intervals in health check
* fix(supply-chain): address all code review findings from PR #2873
- Rename costIncreasePct → supplyDeficitPct (semantic correction)
- Add primaryChokepointWarRiskTier to GetBypassOptionsResponse
- Consolidate ThreatLevel/threatLevelToWarRiskTier into _insurance-tier.ts
- Replace inline CpEntry/ChokepointStatusCacheEntry with ChokepointInfo
- Add outer cachedFetchJson wrapper (3 serial Redis reads → 1 on warm path)
- Add hs2 validation guard matching sibling handler pattern
- Extract CHOKEPOINT_STATUS_KEY constant; eliminate string literal duplication
- Add SCORE_RISK_WEIGHT/SCORE_COST_WEIGHT named constants; clamp liveScore ≥ 0
- Add Math.max(0,...) to liveScore for sub-1.0 cost multiplier corridors
- Fix closurePct: req.closurePct ?? 100 (was || which falsy-coalesced zero)
- Type fetchBypassOptions cargoType as CargoType (was implicit string)
- Add exhaustiveness check to threatLevelToInsurancePremiumBps switch
- Move TIER_RANK to module level in _insurance-tier.ts
- Update WIDGET_PRO_SYSTEM_PROMPT with both new PRO RPCs
* fix(supply-chain): fix supplyDeficitPct averaging and coverageDays sentinel
- Remove .filter(d > 0) from productDeficits: zero-deficit products have demand
and must stay in the denominator to avoid overstating the average
- Clamp coverageDays = Math.max(0, effectiveCoverDays): prevents -1 net-exporter
sentinel from leaking into the public API response
- Update proto comment: document 0 for net exporters
- Add test assertions for both contracts
* chore(api-docs): regenerate OpenAPI docs for coverage_days comment update
* refactor(supply-chain): use CHOKEPOINT_STATUS_KEY in chokepoint-status writer
The key was extracted to cache-keys.ts in the previous commit but the primary
writer (getChokepointStatus) and BOOTSTRAP_CACHE_KEYS still embedded the raw
string literal. Import the constant at both sites to complete the refactor.
* test: update supply-chain-v2 assertions for CHOKEPOINT_STATUS_KEY refactor
Handler now imports CHOKEPOINT_STATUS_KEY as REDIS_CACHE_KEY from cache-keys.ts
rather than defining a local constant. BOOTSTRAP_CACHE_KEYS also references the
constant. Update source-string assertions to match the new patterns.
* fix: keep BOOTSTRAP_CACHE_KEYS.chokepoints as string literal
bootstrap.test.mjs enforces string-literal values in BOOTSTRAP_CACHE_KEYS via
regex. CHOKEPOINT_STATUS_KEY is used in handler imports and is the primary dedup
win; the static registry entry stays as-is per test contract.
* fix(seed): rewrite resilience scores seed as HTTP warm-ping to ranking endpoint
Replaces the tsx/esm scorer import (which failed on Railway due to
rootDirectory=scripts excluding server/) with a single HTTP call to
the Vercel ranking endpoint. The handler's warmMissingResilienceScores
computes all missing scores in-region.
Self-contained .mjs — no TypeScript imports, works with rootDirectory=scripts.
* chore(test): fix misleading test title
* feat(supply-chain): Sprint 0 — chokepoint registry, HS2 sectors, war_risk_tier
- src/config/chokepoint-registry.ts: single source of truth for all 13
canonical chokepoints with displayName, relayName, portwatchName,
corridorRiskName, baselineId, shockModelSupported, routeIds, lat/lon
- src/config/hs2-sectors.ts: static dictionary for all 99 HS2 chapters
with category, shockModelSupported (true only for HS27), cargoType
- server/worldmonitor/supply-chain/v1/_chokepoint-ids.ts: migrated to
derive CANONICAL_CHOKEPOINTS from chokepoint-registry; no data duplication
- src/config/geo.ts + src/types/index.ts: added chokepointId field to
StrategicWaterway interface and all 13 STRATEGIC_WATERWAYS entries
- src/components/MapPopup.ts: switched chokepoint matching from fragile
name.toLowerCase() to direct chokepointId === id comparison
- server/worldmonitor/intelligence/v1/_shock-compute.ts: migrated from old
IDs (hormuz/malacca/babelm) to canonical IDs (hormuz_strait/malacca_strait/
bab_el_mandeb); same for CHOKEPOINT_LNG_EXPOSURE
- proto/worldmonitor/supply_chain/v1/supply_chain_data.proto: added
WarRiskTier enum + war_risk_tier field (field 16) on ChokepointInfo
- get-chokepoint-status.ts: populates warRiskTier from ChokepointConfig.threatLevel
via new threatLevelToWarRiskTier() helper (FREE field, no PRO gate)
* feat(supply-chain): Sprint 1 — country chokepoint exposure index + sector ring
S1.1: scripts/shared/country-port-clusters.json
~130 country → {nearestRouteIds, coastSide} mappings derived from trade route
waypoints; covers all 6 seeded Comtrade reporters plus major trading nations.
S1.2: scripts/seed-hs2-chokepoint-exposure.mjs
Daily cron seeder. Pure computation — reads country-port-clusters.json,
scores each country against CHOKEPOINT_REGISTRY route overlap, writes
supply-chain:exposure:{iso2}:{hs2}:v1 keys + seed-meta (24h TTL).
S1.3: RPC get-country-chokepoint-index (PRO-gated, request-varying)
- proto: GetCountryChokepointIndexRequest/Response + ChokepointExposureEntry
- handler: isCallerPremium gate; cachedFetchJson 24h; on-demand for any iso2
- cache-keys.ts: CHOKEPOINT_EXPOSURE_KEY(iso2, hs2) constant
- health.js: chokepointExposure SEED_META entry (48h threshold)
- gateway.ts: slow-browser cache tier
- service client: fetchCountryChokepointIndex() exported
S1.4: Chokepoint popup HS2 sector ring chart (PRO-gated)
Static trade-sector breakdown (IEA/UNCTAD estimates) per 9 major chokepoints.
SVG donut ring + legend shown for PRO users; blurred lockout + gate-hit
analytics for free users. Wired into renderWaterwayPopup().
🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.com/claude-code) + Compound Engineering v2.49.0
Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* fix(tests): update energy-shock-v2 tests to use canonical chokepoint IDs
CHOKEPOINT_EXPOSURE and CHOKEPOINT_LNG_EXPOSURE keys were migrated from
short IDs (hormuz, malacca, babelm) to canonical registry IDs
(hormuz_strait, malacca_strait, bab_el_mandeb) in Sprint 0.
Test fixtures were not updated at the time; fix them now.
* fix(tests): update energy-shock-seed chokepoint ID to canonical form
VALID_CHOKEPOINTS changed to canonical IDs in Sprint 0; the seed test
that checks valid IDs was not updated alongside it.
* fix(cache-keys): reword JSDoc comment to avoid confusing bootstrap test regex
The comment "NOT in BOOTSTRAP_CACHE_KEYS" caused the bootstrap.test.mjs
regex to match the comment rather than the actual export declaration,
resulting in 0 entries found. Rephrase to "excluded from bootstrap".
* fix(supply-chain): address P1 review findings for chokepoint exposure index
- Add get-country-chokepoint-index to PREMIUM_RPC_PATHS (CDN bypass)
- Validate iso2/hs2 params before Redis key construction (cache injection)
- Fix seeder TTL to 172800s (2× interval) and extend TTL on skipped lock
- Fix CHOKEPOINT_EXPOSURE_SEED_META_KEY to match seeder write key
- Render placeholder sectors behind blur gate (DOM data leakage)
- Document get-country-chokepoint-index in widget agent system prompts
* fix(lint): resolve Biome CI failures
- Add biome.json overrides to silence noVar in HTML inline scripts,
disable linting for public/ vendor/build artifacts and pro-test/
- Remove duplicate NG and MW keys from country-port-clusters.json
- Use import attributes (with) instead of deprecated assert syntax
* fix(build): drop JSON import attribute — esbuild rejects `with` syntax
---------
Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* feat(energy): golden-fixture upstream format tests for 6 seeders (V6-4)
Add static fixture files (CSV/JSON) and golden-fixture test blocks to
catch parser regressions when upstream data formats change. Covers:
OWID energy mix, JODI oil, JODI gas, IEA oil stocks, PortWatch ArcGIS,
and Ember monthly electricity. Also exports buildHistory from
seed-portwatch.mjs for testability.
* fix(tests): pin portwatch fixture lookup by date; remove duplicate fixture test files
* fix(deep-dive): revert prediction category to 'economy' + country-wide nuclear count
Bug 1: PR #2838 changed RPC category from 'economy' to 'finance' based
on a Greptile suggestion, but the server only recognizes 'economy' as a
valid FINANCE_CATEGORY_TAG. 'finance' fell through to the default
geopolitical bucket, returning empty results for US markets.
Bug 2: Infrastructure Exposure showed "Nearby Nuclear: 2" for the US
because it searched within 300km of the country centroid (Kansas).
Added operator (ISO2) field to all 128 nuclear facilities and created
getCountryInfrastructure() which matches by operator code. Countries
with nuclear facilities show country-wide totals (US: 33). Countries
without show nearby facilities from neighboring countries.
* fix(geo): correct 30 nuclear facility operator codes misassigned as 'IN'
The automated script didn't detect country block transitions after
the India section, assigning 'IN' (India) to facilities from Canada,
Hungary, Czech Republic, Sweden, Finland, Belgium, Spain, and 18
other countries. India: 34 → 4 (correct).
* fix(geo): complete operator assignment for all 128 nuclear facilities
Previous automated pass missed facilities in sections with nested
country comments (Germany inside UK block) and left all major country
blocks (US, France, UK, Russia, China, Japan, Korea, etc.) without
operators. Now every facility has the correct ISO2 operator code.
* fix(energy): V6 review findings — flow availability, ember proto bool, LNG stale/blocking, fixture accuracy
- Fix 1: Only show "Flow data unavailable" for chokepoints with PortWatch
flow coverage (hormuz, malacca, suez, bab_el_mandeb), not all corridors
- Fix 2: Correct proto comment on data_available field 9 to document
gas mode and both mode behavior
- Fix 3: Add ember_available bool field 50 to GetCountryEnergyProfile proto,
set server-side from spine.electricity or direct Ember key fallback
- Fix 4: Ember fallback reads energy:ember:v1:{code} when spine exists but
has no electricity block (or fossilShare is absent)
- Fix 6: Add IEA upstream fixture matching actual API response shape,
with golden test parsing through seeder parseRecord/buildIndex
- Fix 7: Add PortWatch ArcGIS fixture with all attributes.* fields used
by buildHistory, with golden test validating parsed output
* fix(energy): add emberAvailable to energy gate; use real buildHistory in portwatch test
* fix(energy): add Ember render block to renderEnergyProfile for Ember-only countries
* chore: regenerate OpenAPI specs after proto comment update
* feat(resilience): populate dataVersion field from seed-meta timestamp
Sets dataVersion to the ISO date of the most recent static bundle
seed, making the data vintage visible to API consumers.
* fix(resilience): bump score cache to v7 for dataVersion field addition
* fix(energy): remove MULTI/EXEC from Ember pipeline (unsupported by Upstash REST); remove redundant ais-relay Ember loop
* test(energy): remove duplicate pipeline failure detection describe block
Lines 289-301 were a strict subset of the block at lines 235-253,
causing duplicate test entries in reports.
* fix(panel): sectionCard ? tooltip + Energy Profile CMD+K entry
- Add optional helpText param to sectionCard(); renders a button.cdp-card-help with title attribute
- Pass tooltip text for Energy Profile and Maritime Activity call sites
- Add country:energy-profile CMD+K entry after maritime-activity in commands.ts
- Update country-port-activity test regex to match new sectionCard call signature
* fix(cmdK): correct energy-profile command ID prefix and add cdp-card-help styles
- Change command id from country:energy-profile to panel:energy-profile to avoid
ISO code lookup collision in search-manager.ts
- Add .cdp-card-help CSS rule for the ? tooltip badge using existing CSS custom
properties (var(--text-muted), currentColor) to match panel design system
* perf(cache): add CF edge caching, eliminate request storms
Three changes to reduce origin request volume:
1. Gateway cache tiers now include `public, s-maxage` so Cloudflare
can cache API responses at edge (previously browser-only). Bumped
27 slow-seeded endpoints to appropriate tiers (static->daily for
6h+ seeds, slow->static for 2h seeds).
2. Population exposure: moved computation client-side. The server
handler is pure math on 20 hardcoded countries, no reason for
network calls. Eliminates ~17.7M requests/week (20 calls per
page load -> 0).
3. Consumer prices: wrapped fetchAllMarketsOverview in a circuit
breaker so the combined 8-market result is cached as a unit.
Returning visitors within 30min hit localStorage instead of
firing 8 separate API calls.
* test: update shipping-rates tier assertion (static -> daily)
* test: update cache tier assertions for three-tier caching design
* fix(security): force slow-browser tier for premium endpoints
Premium endpoints (PREMIUM_RPC_PATHS + ENDPOINT_ENTITLEMENTS) must not
get public s-maxage headers. CF would cache authenticated responses and
serve them without re-running auth/entitlement checks. Force these to
slow-browser tier (browser-only max-age, no public/s-maxage).
* fix(security): add list-market-implications to PREMIUM_RPC_PATHS
PRO-only panel endpoint was missing from premium paths, allowing CF
edge caching to serve authenticated responses to unauthenticated users.
* chore: disable deduct-situation panel and endpoint
Panel set to enabled:false in panels.ts, server handler returns
early with provider:'disabled'. Code preserved for re-enabling later.
* fix(security): suppress CDN-Cache-Control for premium endpoints too
P1: slow-browser tier still had CDN-Cache-Control with public s-maxage,
letting Vercel CDN cache premium responses for same-origin requests.
Now CDN caching is fully disabled for premium endpoints.
P2: revert server-side deduct-situation disable. Keep backend intact
so the published API and correlation engine enrichment still work.
Only the panel is disabled (enabled:false in panels.ts).
* feat(energy): per-product refinery yield coefficients replacing 0.8 heuristic (V5-5)
* fix(energy): yield scales crudeLossKbd not demand; worst deficit across all products
* fix(energy): update seed test to match worst-product assessment text
* feat(energy): LNG gas shock path with GIE storage buffer (V5-4)
* fix(energy): gas path zero-LNG handling, oil gate bypass, live flow scaling, coverage semantics
* fix(energy): gas-only coverage respects degraded; zero oil fields in gas mode
* feat(resilience): baseline vs stress scoring engine
Splits the resilience index into structural capacity (baselineScore)
and active disruption (stressScore) using the dimension type tags from
RESILIENCE_DIMENSION_TYPES (baseline/stress/mixed).
overallScore = baselineScore * (1 - stressFactor) where stressFactor
is clamped to [0, 0.5]. Mixed dimensions contribute to both scores.
Proto fields 10-12 added (baseline_score, stress_score, stress_factor).
Widget updated to display baseline/stress breakdown.
Cache keys bumped v4 -> v5 for atomic rollout.
* fix(resilience): bump history key to v2 for baseline/stress formula change
The overallScore formula changed from domain-weighted-sum to
baselineScore * (1 - stressFactor). Old history entries are
incomparable, causing fake change30d drops of -20 to -30 points.
Versioned history key starts a clean series.
* feat(energy): shock model v2 — live flow ratios, coverage, limitations (V5-3)
Replace static CHOKEPOINT_EXPOSURE multipliers with live PortWatch flow
ratios from energy:chokepoint-flows:v1. Add proto fields 11-19: per-source
coverage flags (jodi_oil_coverage, comtrade_coverage, iea_stocks_coverage,
portwatch_coverage), coverage_level, limitations[], degraded,
chokepoint_confidence, live_flow_ratio. Expand ISO2_TO_COMTRADE from 6 to
150+ countries via _comtrade-reporters.ts. Partial-coverage path proxies
Gulf share at 40%. Unsupported countries return structured metadata instead
of opaque string. data_available (field 9) preserved for backward compat.
* fix(energy): correct chokepoint-flows key shape in shock handler (V5-3)
energy:chokepoint-flows:v1 is a flat object keyed by canonicalId
(hormuz_strait, bab_el_mandeb, suez, malacca_strait), not an object
with a chokepoints[] array. The wrong shape caused degraded=true and
liveFlowRatio=null for every request, silently falling back to static
CHOKEPOINT_EXPOSURE multipliers even when live PortWatch data was
available.
Fixes: ChokepointEntry interface, typecast, cpEntry lookup, degraded check.
* fix(energy): 4 review fixes for shock v2 handler (V5-3)
1. Cache key v1→v2: response shape changed (fields 11-19 added), old
v1 cache entries would be served without new coverage fields.
2. IEA unknown vs zero: when ieaStocksCoverage=false, assessment now
shows "IEA cover: unknown" instead of "0 days" to avoid conflating
missing data with real zero stock.
3. liveFlowRatio 0.0 truthiness: changed `if (liveFlowRatio)` to
`if (liveFlowRatio != null)` — a blocked chokepoint (ratio=0.0)
is now shown, not hidden as "no live data".
4. Badge cleared on request start: coverageBadge is now reset before
each shock compute request so a failed request doesn't leave the
previous result's badge visible.
* fix(energy): 3 remaining review fixes for shock v2 (V5-3)
1. Flow column: gate on portwatchCoverage (bool) not liveFlowRatio!=null —
proto double defaults to 0, so null-check was always true and degraded
responses showed a misleading "Flow: 0%" column.
2. Degraded cache TTL: 5min when degraded=true, 1h when live data present —
limits how long stale degraded state persists after PortWatch recovers.
3. IEA anomaly cover days: zero daysOfCover when ieaStocksCoverage=false
so anomalous IEA data no longer contributes to effectiveCoverDays;
panel IEA cover row also gated on ieaStocksCoverage.
* fix(energy): netExporter from anomalous IEA + zero-days panel display (V5-3)
1. netExporter: gated on ieaStocksCoverage — anomalous IEA rows no
longer drive the net-exporter assessment branch when coverage=false.
2. Panel IEA cover: removed effectiveCoverDays>0 guard so a real zero
(reserves exhausted under scenario) renders as "0 days" instead of
being silently hidden as if there were no IEA data.
* fix(energy): handle net-exporter sentinel (-1) in IEA cover panel row
* fix(energy): NaN guard on flowRatio; optional live_flow_ratio; coverageLevel includes IEA+degraded
* fix(energy): narrow Gulf-share proxy to Comtrade-only; NaN guard computeGulfShare; EMPTY liveFlowRatio undefined
* fix(energy): tighten ieaStocksCoverage null guard; cache key varies by degraded state
* fix(energy): harden IEA/PortWatch input validation; reduce shock cache TTL
* fix(energy): add null narrowing for daysOfCover to satisfy strict TS
* feat(energy): Ember monthly electricity seed — V5-6a
New seed-ember-electricity.mjs writes energy:ember:v1:<ISO2> and
energy:ember:v1:_all from Ember Climate's monthly generation CSV (CC BY 4.0).
Daily cron at 08:00 UTC, TTL 72h (3x interval), >=60 country coverage guard.
Registers in api/health.js, api/seed-health.js, cache-keys.ts, and
ais-relay.cjs. Dockerfile.relay COPY added.
🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.ai/code) + Compound Engineering v2.49.0
Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* fix(energy): add _country-resolver.mjs to Dockerfile.relay; correct Ember intervalMin (V5-6a)
Two bugs in the Ember seed PR:
1. Dockerfile.relay was missing COPY for _country-resolver.mjs, which
seed-ember-electricity.mjs imports. Would have crashed with
ERR_MODULE_NOT_FOUND on first run in production.
2. api/seed-health.js had intervalMin:720 (12h) for a daily (24h) cron.
With stale threshold = intervalMin*2, this gave only 24h grace --
the seed would flap stale during the CSV download window.
Corrected to intervalMin:1440 so stale threshold = 48h (2x interval).
* fix(energy): wire energy:ember:v1:_all into bootstrap hydration (V5-6a)
Greptile P1: api/bootstrap.js was missing the emberElectricity slow-key
entry, violating the AGENTS.md requirement that new data sources be
registered for bootstrap hydration.
energy:ember:v1:_all is a ~60-country bulk map (monthly cadence) -
added to SLOW_KEYS consistent with faoFoodPriceIndex and other
monthly-release bulk keys.
Also updates server/_shared/cache-keys.ts BOOTSTRAP_CACHE_KEYS and
BOOTSTRAP_TIERS to keep the bootstrap test coverage green (bootstrap
test validates that SLOW_KEYS and BOOTSTRAP_TIERS are in sync).
* fix(energy): 3 review fixes for Ember seed (V5-6a)
1. Ember URL: updated to correct current download URL (old path
returned HTTP 404, seeder could never run).
2. Count-drop guard after failure: failure path now preserves the
previous recordCount in seed-meta instead of writing 0, so the
75% drop guard stays active after a failed run.
3. api/seed-health.js: status:error now marks seed as stale/error
immediately instead of only checking age; prevents /api/seed-health
showing ok for 48h while the seeder is failing.
* fix(energy): correct Ember CSV column names + fix skipped-path meta (V5-6a)
1. CSV schema: parser was using country_code/series/unit/value/date
but the real Ember CSV headers are "ISO 3 code"/"Variable"/"Unit"/
"Value"/"Date". Added COLS constants and updated all row field
accesses. The schema sentinel (hasFossil check) was always firing
because r.series was always undefined, causing every seeder run to
abort. Updated test fixtures to use real column names.
2. Skipped-path meta: lock.skipped branch now reads existing meta and
preserves recordCount and status while refreshing fetchedAt.
Previously writing recordCount:0 disabled the count-drop guard after
any skipped run and made health endpoints see false-ok with zero count.
* fix(energy): remove skipped-path meta write + revert premature bootstrap (V5-6a)
1. lock.skipped: removed seed-meta write from the skipped path. The
running instance writes correct meta on completion; refreshing
fetchedAt on skip masked relay/lock failures from health endpoints.
2. Bootstrap: removed emberElectricity from BOOTSTRAP_CACHE_KEYS and
BOOTSTRAP_TIERS — no consumer exists in src/ yet. Per energyv5.md,
bootstrap registration is deferred to PR7 when consumers land.
* fix(energy): split ember pipeline writes; fix health.js recordCount lookup
- api/health.js: add recordCount fallback in both seed-meta count reads so
the Ember domain shows correct record count instead of always 1
- scripts/seed-ember-electricity.mjs: split single pipeline into Phase A
(per-country + _all data) and Phase B (seed-meta only after Phase A
succeeds) to prevent preservePreviousSnapshot reading a partial _all key
* fix(energy): split ember pipeline writes; align SEED_ERROR in health.js; add tests
* fix(energy): atomic rollback on partial pipeline failure; seedError priority in health cascade
* fix(energy): DEL obsolete per-country keys on publish, rollback, and restore
* fix(energy): MULTI/EXEC atomic pipeline; null recordCount on read-miss; dataWritten guard
---------
Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* feat(email): add deliverability guards to reduce waitlist bounces
Analyzed 1,276 bounced waitlist emails and found typos (gamil.com),
disposable domains (passmail, guerrillamail), offensive submissions,
and non-existent domains account for the majority.
Four layers of protection:
- Frontend: mailcheck.js typo suggestions on email blur
- API: MX record check via Cloudflare DoH, disposable domain
blocklist, offensive pattern filter, typo-TLD blocklist
- Webhook: api/resend-webhook.js captures bounce/complaint events,
stores in Convex emailSuppressions table, checked before sending
- Tooling: import script for bulk-loading existing bounced emails
* fix(email): address review - auth, retry, CSV parsing
1. Security: Convert suppress/bulkSuppress/remove to internalMutation.
Webhook now runs as Convex httpAction (matching Dodo pattern) with
direct access to internal mutations. Bulk import uses relay shared
secret. Only isEmailSuppressed remains public (read-only query).
2. Retry: Convex httpAction returns 500 on any mutation failure so
Resend retries the webhook instead of silently losing bounce events.
3. CSV: Replace naive comma-split with RFC 4180 parser that handles
quoted fields. Import script now calls Convex HTTP action
authenticated via RELAY_SHARED_SECRET instead of public mutation.
* fix(email): make isEmailSuppressed internal, check inside mutation
Move suppression check into registerInterest:register mutation
(same transaction, no extra round-trip). Remove public query
entirely so no suppression data is exposed to browser clients.
* test(email): add coverage for validation, CSV parser, and suppressions
- 19 tests for validateEmail: disposable domains, offensive patterns,
typo TLDs, MX fail-open, case insensitivity, privacy relay allowance
- 7 tests for parseCsvLine: RFC 4180 quoting, escaped quotes, empty
fields, Resend CSV format with angle brackets and commas
- 11 Convex tests for emailSuppressions: suppress idempotency, case
normalization, bulk dedup, remove, and registerInterest integration
(emailSuppressed flag in mutation return)
* feat(resilience): add WB mean applied tariff rate to tradeSanctions
World Bank TM.TAX.MRCH.WM.AR.ZS covers 180+ countries, supplementing
the WTO top-50 metrics that only cover major reporters. Reduces
reporter-set bias by providing a global trade openness signal.
Reweights: sanctions 0.45, WTO restrictions 0.15, WTO barriers 0.15,
WB tariff rate 0.25.
* fix: update pinned test assertions for WB tariff rate reweighting
Adjusts scoreTradeSanctions test assertions for the new 4-metric blend
(sanctions 0.45, restrictions 0.15, barriers 0.15, tariff 0.25) and
bumps TOTAL_DATASET_SLOTS from 9 to 10 in payload assembly tests.
* fix(seed): bump static source version to v5 + sync indicator registry for trade
Version bump ensures appliedTariffRate backfills to existing 2026
snapshots. Registry updated from 3-metric to 4-metric trade-sanctions
weights.
* fix(resilience): correct appliedTariffRate sourceKey to resilience:static:{ISO2}
* fix(resilience): bump score cache to v4 + add tariff rate to release-gate fixtures
Score/ranking cache keys bumped to v4 to invalidate stale pre-tariff
cached responses. Release-gate fixtures now include appliedTariffRate
so the gate exercises the full 4-metric trade-sanctions path.
* fix(test): update pinned scorer assertions after rebase onto main
With all Phase 2+3 PRs merged (FX reserves, broadband, WHO metrics,
zero-event guards), the combined fixture data produces economic=66.33,
infrastructure=79, overallScore=68.72.
* feat(resilience): add FX reserves adequacy to currencyExternal dimension
World Bank FI.RES.TOTL.MO (total reserves in months of imports) covers
~160 countries, filling the BIS EER coverage gap (~40 economies).
For BIS countries: reserves supplement volatility + deviation (weight 0.15).
For non-BIS countries: reserves combine with IMF inflation proxy (0.4/0.6
blend) for much better currency stability coverage than inflation alone.
Normalization: 1 month (near crisis) = 0, 12+ months (very safe) = 100.
* fix(seed): bump static source version to v4 for fxReservesMonths backfill
Without version bump, existing 2026 snapshots won't be republished and
fxReservesMonths field will never backfill until next annual cycle.
* fix(resilience): bump score cache to v3 for FX reserves scorer change
scoreCurrencyExternal now includes FX reserves adequacy, changing scores
for all countries. Bump cache key to invalidate stale pre-reserves
cached responses on deploy.
* fix(seed): retry static seed when previous run had failed datasets
shouldSkipSeedYear() now returns false when seed-meta records non-empty
failedDatasets, allowing backfill of datasets that failed on the first
run (e.g., fxReservesMonths upstream outage during v4 rollout).
Previously, partial success with status:'ok' caused all future same-year
runs to skip permanently.
* feat(resilience): add WB broadband penetration to infrastructure dimension
World Bank IT.NET.BBND.P2 (fixed broadband subscriptions per 100
people) added as new sub-metric in scoreInfrastructure.
normalizeHigherBetter(0, 40). Reweights: electricity 0.30, roads 0.30,
outages 0.25, broadband 0.15.
* fix(resilience): add explicit outagesRaw null guard in scoreInfrastructure
Matches the established pattern in scoreCyberDigital where both source
presence and penalty > 0 are checked before scoring.
* test(resilience): pin expected broadband numeric contribution in infrastructure scorer
Strengthens the broadband test from directional-only to pinned numeric
assertion, catching regressions in normalization goalposts or weight
changes.
* feat(resilience): add WHO physician density + health expenditure to healthPublicService
Two new sub-metrics from WHO GHO OData:
- Physician density per 1k (HWF_0001): normalizeHigherBetter(0, 5)
- Health expenditure per capita PPP (GHED_CHE_pc_PPP_SHA2011): normalizeHigherBetter(50, 5000)
Reweights existing metrics: UHC 0.35, measles 0.25, beds 0.10,
physicians 0.15, expenditure 0.15.
Bumps static source version to v3 for backfill.
* fix(resilience): replace dead WHO health expenditure indicator with working alternative
GHED_CHE_pc_PPP_SHA2011 returns empty dataset from WHO GHO API, causing
the 15% healthExp weight to silently drop from production scoring.
Replaced with GHED_CHE_pc_US_SHA2011 (per capita current USD), which has
4849 records across all countries. Renamed field healthExpPerCapitaPpp to
healthExpPerCapitaUsd and adjusted normalization goalposts from (50, 5000)
to (20, 3000) to reflect current-USD scale. Bumped source version to v4.
* fix(seed): increase WHO $top to 10000 to prevent pagination truncation + add transform test
WHO GHO API returns exactly 1000 rows with no @odata.nextLink for
physician density and health expenditure indicators, silently truncating
country coverage. Increasing $top to 10000 fetches all rows in one page
(typical WHO indicators have 2000-5000 rows).
Also adds seed-level test for the HWF_0001 per-10k to per-1k division
transform.
