* feat(supply-chain): Sprint 0 — chokepoint registry, HS2 sectors, war_risk_tier
- src/config/chokepoint-registry.ts: single source of truth for all 13
canonical chokepoints with displayName, relayName, portwatchName,
corridorRiskName, baselineId, shockModelSupported, routeIds, lat/lon
- src/config/hs2-sectors.ts: static dictionary for all 99 HS2 chapters
with category, shockModelSupported (true only for HS27), cargoType
- server/worldmonitor/supply-chain/v1/_chokepoint-ids.ts: migrated to
derive CANONICAL_CHOKEPOINTS from chokepoint-registry; no data duplication
- src/config/geo.ts + src/types/index.ts: added chokepointId field to
StrategicWaterway interface and all 13 STRATEGIC_WATERWAYS entries
- src/components/MapPopup.ts: switched chokepoint matching from fragile
name.toLowerCase() to direct chokepointId === id comparison
- server/worldmonitor/intelligence/v1/_shock-compute.ts: migrated from old
IDs (hormuz/malacca/babelm) to canonical IDs (hormuz_strait/malacca_strait/
bab_el_mandeb); same for CHOKEPOINT_LNG_EXPOSURE
- proto/worldmonitor/supply_chain/v1/supply_chain_data.proto: added
WarRiskTier enum + war_risk_tier field (field 16) on ChokepointInfo
- get-chokepoint-status.ts: populates warRiskTier from ChokepointConfig.threatLevel
via new threatLevelToWarRiskTier() helper (FREE field, no PRO gate)
* feat(supply-chain): Sprint 1 — country chokepoint exposure index + sector ring
S1.1: scripts/shared/country-port-clusters.json
~130 country → {nearestRouteIds, coastSide} mappings derived from trade route
waypoints; covers all 6 seeded Comtrade reporters plus major trading nations.
S1.2: scripts/seed-hs2-chokepoint-exposure.mjs
Daily cron seeder. Pure computation — reads country-port-clusters.json,
scores each country against CHOKEPOINT_REGISTRY route overlap, writes
supply-chain:exposure:{iso2}:{hs2}:v1 keys + seed-meta (24h TTL).
S1.3: RPC get-country-chokepoint-index (PRO-gated, request-varying)
- proto: GetCountryChokepointIndexRequest/Response + ChokepointExposureEntry
- handler: isCallerPremium gate; cachedFetchJson 24h; on-demand for any iso2
- cache-keys.ts: CHOKEPOINT_EXPOSURE_KEY(iso2, hs2) constant
- health.js: chokepointExposure SEED_META entry (48h threshold)
- gateway.ts: slow-browser cache tier
- service client: fetchCountryChokepointIndex() exported
S1.4: Chokepoint popup HS2 sector ring chart (PRO-gated)
Static trade-sector breakdown (IEA/UNCTAD estimates) per 9 major chokepoints.
SVG donut ring + legend shown for PRO users; blurred lockout + gate-hit
analytics for free users. Wired into renderWaterwayPopup().
🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.com/claude-code) + Compound Engineering v2.49.0
Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* fix(tests): update energy-shock-v2 tests to use canonical chokepoint IDs
CHOKEPOINT_EXPOSURE and CHOKEPOINT_LNG_EXPOSURE keys were migrated from
short IDs (hormuz, malacca, babelm) to canonical registry IDs
(hormuz_strait, malacca_strait, bab_el_mandeb) in Sprint 0.
Test fixtures were not updated at the time; fix them now.
* fix(tests): update energy-shock-seed chokepoint ID to canonical form
VALID_CHOKEPOINTS changed to canonical IDs in Sprint 0; the seed test
that checks valid IDs was not updated alongside it.
* fix(cache-keys): reword JSDoc comment to avoid confusing bootstrap test regex
The comment "NOT in BOOTSTRAP_CACHE_KEYS" caused the bootstrap.test.mjs
regex to match the comment rather than the actual export declaration,
resulting in 0 entries found. Rephrase to "excluded from bootstrap".
* fix(supply-chain): address P1 review findings for chokepoint exposure index
- Add get-country-chokepoint-index to PREMIUM_RPC_PATHS (CDN bypass)
- Validate iso2/hs2 params before Redis key construction (cache injection)
- Fix seeder TTL to 172800s (2× interval) and extend TTL on skipped lock
- Fix CHOKEPOINT_EXPOSURE_SEED_META_KEY to match seeder write key
- Render placeholder sectors behind blur gate (DOM data leakage)
- Document get-country-chokepoint-index in widget agent system prompts
* fix(lint): resolve Biome CI failures
- Add biome.json overrides to silence noVar in HTML inline scripts,
disable linting for public/ vendor/build artifacts and pro-test/
- Remove duplicate NG and MW keys from country-port-clusters.json
- Use import attributes (with) instead of deprecated assert syntax
* fix(build): drop JSON import attribute — esbuild rejects `with` syntax
---------
Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* feat(flights): add adsb.lol /v2/mil as military flight fallback
Adds adsb.lol community ADS-B military feed as second fallback in the
theater posture seeder, between OpenSky (primary) and Wingbits (last
resort). Free, no auth, pre-filtered military aircraft (dbFlags bit 1).
Filters to theater bounding boxes and produces same flight shape.
Fallback chain: OpenSky → adsb.lol → Wingbits → vessel-only posture
* fix(flights): distinguish empty vs error in adsb.lol fallback + regex tilde strip
- null = fetch error → fall through to Wingbits
- [] = successful but no theater traffic → stop (don't call Wingbits)
- Use /~/g regex for ICAO hex tilde removal (handles multiple occurrences)
* feat(oref): add Tzeva Adom as primary alert source with Hebrew translations
Tzeva Adom API (api.tzevaadom.co.il/notifications) is now the primary
source for Israeli siren alerts. Free, no proxy needed, works from any
IP. OREF direct (via residential proxy) remains as fallback.
Includes Hebrew→English translation dictionaries:
- 1,305 Israeli localities from CBS data
- 28 threat type terms (missiles, rockets, drones, infiltration, etc.)
Alert titles and location names are now served in English.
Fallback chain: Tzeva Adom (primary) → OREF direct (proxy, fallback)
* fix(oref): add threat categorization + misplaced-city guard to Tzeva Adom
- categorizeOrefThreat() classifies Hebrew/English alerts into
MISSILE/ROCKET/DRONE/MORTAR/INFILTRATION/EARTHQUAKE/TSUNAMI/HAZMAT
- Detects when API puts a city name in the threat field and moves it
to locations (known API quirk)
* fix(oref): decouple siren poll loop from OREF proxy availability
SIREN_ALERTS_ENABLED is always true (Tzeva Adom needs no proxy).
OREF_PROXY_AVAILABLE gates only the OREF fallback path.
The poll loop now starts regardless of proxy config, using Tzeva Adom
as primary and OREF as fallback only when OREF_PROXY_AUTH is set.
Response payloads report configured: true so the panel activates.
* fix(oref): preserve error state when both siren sources fail
When Tzeva Adom returns null and OREF proxy is unavailable, return
early with lastError set instead of falling through and clearing
the error. Prevents a false green state in the panel when both
sources are down.
* fix(oref): rebuild response cache on source outage
Without calling orefPreSerializeResponses() in the failure branch,
the /oref/alerts handler keeps serving stale _alertsCache from the
last successful poll, masking the outage.
* fix(energy): remove MULTI/EXEC from Ember pipeline (unsupported by Upstash REST); remove redundant ais-relay Ember loop
* test(energy): remove duplicate pipeline failure detection describe block
Lines 289-301 were a strict subset of the block at lines 235-253,
causing duplicate test entries in reports.
* feat(energy): Ember monthly electricity seed — V5-6a
New seed-ember-electricity.mjs writes energy:ember:v1:<ISO2> and
energy:ember:v1:_all from Ember Climate's monthly generation CSV (CC BY 4.0).
Daily cron at 08:00 UTC, TTL 72h (3x interval), >=60 country coverage guard.
Registers in api/health.js, api/seed-health.js, cache-keys.ts, and
ais-relay.cjs. Dockerfile.relay COPY added.
🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.ai/code) + Compound Engineering v2.49.0
Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* fix(energy): add _country-resolver.mjs to Dockerfile.relay; correct Ember intervalMin (V5-6a)
Two bugs in the Ember seed PR:
1. Dockerfile.relay was missing COPY for _country-resolver.mjs, which
seed-ember-electricity.mjs imports. Would have crashed with
ERR_MODULE_NOT_FOUND on first run in production.
2. api/seed-health.js had intervalMin:720 (12h) for a daily (24h) cron.
With stale threshold = intervalMin*2, this gave only 24h grace --
the seed would flap stale during the CSV download window.
Corrected to intervalMin:1440 so stale threshold = 48h (2x interval).
* fix(energy): wire energy:ember:v1:_all into bootstrap hydration (V5-6a)
Greptile P1: api/bootstrap.js was missing the emberElectricity slow-key
entry, violating the AGENTS.md requirement that new data sources be
registered for bootstrap hydration.
energy:ember:v1:_all is a ~60-country bulk map (monthly cadence) -
added to SLOW_KEYS consistent with faoFoodPriceIndex and other
monthly-release bulk keys.
Also updates server/_shared/cache-keys.ts BOOTSTRAP_CACHE_KEYS and
BOOTSTRAP_TIERS to keep the bootstrap test coverage green (bootstrap
test validates that SLOW_KEYS and BOOTSTRAP_TIERS are in sync).
* fix(energy): 3 review fixes for Ember seed (V5-6a)
1. Ember URL: updated to correct current download URL (old path
returned HTTP 404, seeder could never run).
2. Count-drop guard after failure: failure path now preserves the
previous recordCount in seed-meta instead of writing 0, so the
75% drop guard stays active after a failed run.
3. api/seed-health.js: status:error now marks seed as stale/error
immediately instead of only checking age; prevents /api/seed-health
showing ok for 48h while the seeder is failing.
* fix(energy): correct Ember CSV column names + fix skipped-path meta (V5-6a)
1. CSV schema: parser was using country_code/series/unit/value/date
but the real Ember CSV headers are "ISO 3 code"/"Variable"/"Unit"/
"Value"/"Date". Added COLS constants and updated all row field
accesses. The schema sentinel (hasFossil check) was always firing
because r.series was always undefined, causing every seeder run to
abort. Updated test fixtures to use real column names.
2. Skipped-path meta: lock.skipped branch now reads existing meta and
preserves recordCount and status while refreshing fetchedAt.
Previously writing recordCount:0 disabled the count-drop guard after
any skipped run and made health endpoints see false-ok with zero count.
* fix(energy): remove skipped-path meta write + revert premature bootstrap (V5-6a)
1. lock.skipped: removed seed-meta write from the skipped path. The
running instance writes correct meta on completion; refreshing
fetchedAt on skip masked relay/lock failures from health endpoints.
2. Bootstrap: removed emberElectricity from BOOTSTRAP_CACHE_KEYS and
BOOTSTRAP_TIERS — no consumer exists in src/ yet. Per energyv5.md,
bootstrap registration is deferred to PR7 when consumers land.
* fix(energy): split ember pipeline writes; fix health.js recordCount lookup
- api/health.js: add recordCount fallback in both seed-meta count reads so
the Ember domain shows correct record count instead of always 1
- scripts/seed-ember-electricity.mjs: split single pipeline into Phase A
(per-country + _all data) and Phase B (seed-meta only after Phase A
succeeds) to prevent preservePreviousSnapshot reading a partial _all key
* fix(energy): split ember pipeline writes; align SEED_ERROR in health.js; add tests
* fix(energy): atomic rollback on partial pipeline failure; seedError priority in health cascade
* fix(energy): DEL obsolete per-country keys on publish, rollback, and restore
* fix(energy): MULTI/EXEC atomic pipeline; null recordCount on read-miss; dataWritten guard
---------
Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com>
* feat(energy): canonical energy spine seeder + handler read-through (V5-1)
- Add scripts/seed-energy-spine.mjs: daily seeder that assembles one
energy:spine:v1:<ISO2> key per country from 6 domain keys (OWID mix,
JODI oil, JODI gas, IEA stocks, ENTSO-E electricity, GIE gas storage)
- TTL 172800s (48h), count-drop guard at 80%, schema sentinel for OWID mix,
lock pattern + Redis pipeline batch write mirroring seed-owid-energy-mix.mjs
- Update get-country-energy-profile.ts to read from spine first; fall back
to existing 6-key Promise.allSettled join on spine miss
- Update chat-analyst-context.ts buildProductSupply/buildGasFlows/buildOilStocksCover
to prefer spine key; fall through to direct domain key reads on miss
- Update get-country-intel-brief.ts to read energy mix from spine.mix + sources.mixYear
before falling back to energy:mix:v1: direct key
- Add ENERGY_SPINE_KEY_PREFIX and ENERGY_SPINE_COUNTRIES_KEY to cache-keys.ts
- Add energySpineCountries to api/health.js STANDALONE_KEYS and SEED_META
- Add Railway cron comment (0 6 * * *) in ais-relay.cjs
- Add tests/energy-spine-seed.test.mjs: 26 tests covering spine build logic,
IEA anomaly guard, JODI oil fallback, schema sentinel, count-drop math
* fix(energy): add cache-keys module replacement in redis-caching test
The new ENERGY_SPINE_KEY_PREFIX import in get-country-intel-brief.ts
was not patched in the importPatchedTsModule call used by redis-caching
tests. Add cache-keys to the replacement map to resolve the module.
* fix(energy): add missing fields to spine entry and read-through
- buildOilFields: add product importsKbd (gasoline/diesel/jet/lpg) and belowObligation
- buildMixFields: add windShare, solarShare, hydroShare
- buildGasStorageFields: new helper storing fillPct, fillPctChange1d, trend
- buildSpineEntry: add gasStorage section using new helper
- EnergySpine interface: extend oil/mix/gasStorage to match seeder output
- buildResponseFromSpine: read all new fields instead of hard-coding 0/false
* fix(energy): exclude electricity/gas-storage from spine; add seed-health entry
Spine-first path was returning stale gas-storage and electricity data for
up to 8h after seeding (spine runs 06:00 UTC, gas storage updates 10:30 UTC,
electricity updates 14:00 UTC).
Fix: handler now reads gas-storage and electricity directly in parallel with
the spine read (3-key allSettled). Fallback path drops from 6 to 4 keys since
gas-storage and electricity are already fetched in the hot path.
Also registers energy:spine in api/seed-health.js (daily cron, maxStaleMin
inferred as 2×intervalMin = 2880 min).
Seeder (seed-energy-spine.mjs) and its tests updated to reflect the narrowed
spine schema — electricity and gasStorage fields removed from buildSpineEntry.
* fix(energy): address Greptile P2 review findings
- chat-analyst-context: return undefined when gas imports are 0 rather
than falling back to totalDemandTj with an "imports" label — avoids
mislabeling domestic gas demand as imports for net exporters (RU, QA, etc.)
- seed-energy-spine: add status:'ok' to success-path seed-meta write so
all seed-meta records have a consistent status field regardless of path
* feat(energy): chokepoint flow calibration seeder — V5-2 (Phase 4 PR A)
- Add CHOKEPOINT_FLOWS_KEY to server/_shared/cache-keys.ts
- Add energy:chokepoint-flows:v1 to health.js monitoring (maxStaleMin: 720)
- Add 6h chokepoint flow seed loop to ais-relay.cjs (seed-chokepoint-flows.mjs)
- Fix seeder to use degraded mode instead of throwing when PortWatch absent
- Add degraded-mode and ID-mapping tests to chokepoint-flows-seed.test.mjs
* fix(energy): restore throw for PortWatch absent + register seed-health
- seed-chokepoint-flows: revert degraded-path from warn+return{} back to
throw; PortWatch absent is an upstream-not-ready error, not a data-quality
issue — must throw so startChokepointFlowsSeedLoop schedules 20-min retry
- api/seed-health.js: add energy:chokepoint-flows to SEED_DOMAINS so
/api/seed-health surfaces missing/stale signal (intervalMin: 360 = 6h cron)
- tests: update degraded-mode assertions to match restored throw behavior
* feat(supply-chain): move PortWatch seeding to standalone seed-portwatch.mjs
Move PortWatch ArcGIS data fetching out of ais-relay.cjs into a new
standalone Railway cron seeder. Expands vessel type coverage from the
old n_tanker/n_cargo/n_total aggregate to all five individual types
(container, dry bulk, general cargo, RoRo, tanker) plus five DWT
capacity fields per chokepoint.
The relay no longer fetches from ArcGIS — it reads supply_chain:portwatch:v1
from Redis as before. seedTransitSummaries() hydration path is unchanged.
* fix(supply-chain): always read PortWatch from Redis in seedTransitSummaries
Remove the latestPortwatchData in-memory cache guard. With the seed loop
moved to standalone seed-portwatch.mjs, a warm relay would never re-read
from Redis after cold-start hydration, causing stale history and
wowChangePct to be republished into transit-summaries:v1 indefinitely.
seedTransitSummaries now reads supply_chain:portwatch:v1 fresh from
Redis on every 10-minute cycle, picking up the standalone seeder's 6h
refreshes immediately.
* fix(relay): add proxy fallback for Spending/GSCPI, fix OpenSky TLS
- Spending (USASpending.gov): add proxy fallback for POST requests
- GSCPI (NY Fed CSV): add proxy fallback
- OpenSky: remove tls=false override that broke Decodo TLS tunnel
(EPROTO: packet length too long). Decodo always requires TLS.
- Same TLS fix in seed-military-flights.mjs
* fix(relay): force tls=true on Spending/GSCPI proxy calls
parseProxyConfig defaults tls=true for bare strings, but if PROXY_URL
uses http:// scheme it would be false. Force tls=true to match all
other proxy calls in the relay (Decodo always requires TLS).
* refactor: consolidate 5 proxy tunnel implementations into _proxy-utils.cjs
5 near-identical HTTP CONNECT proxy tunnel implementations (3 in
ais-relay.cjs, 1 in _seed-utils.mjs, 1 in seed-military-flights.mjs)
consolidated into two shared functions in _proxy-utils.cjs:
- proxyConnectTunnel(): low-level CONNECT + TLS wrapping, returns socket
- proxyFetch(): high-level fetch with decompression, custom headers,
POST support, timeout
All consumers now call the shared implementation:
- _seed-utils.mjs httpsProxyFetchRaw: 75 lines -> 6 lines
- ais-relay.cjs ytFetchViaProxy: 40 lines -> 5 lines
- ais-relay.cjs _openskyProxyConnect: 35 lines -> 8 lines
- ais-relay.cjs inline Dodo CONNECT: 25 lines -> 10 lines
- seed-military-flights.mjs proxyFetchJson: 70 lines -> 14 lines
Also wires weather alerts proxy fallback (fixes STALE_SEED health crit).
Net: -104 lines. Resolves the TODO at _seed-utils.mjs:311.
* fix(proxy): default tls=true for bare proxy strings
parseProxyConfig returned no tls field for bare-format proxies
(user:pass@host:port and host:port:user:pass). proxyConnectTunnel
checked proxyConfig.tls and used plain TCP when it was undefined,
breaking connections to Decodo which requires TLS. Only http:// URLs
should use plain TCP.
* fix(proxy): timeout covers full response, pass targetPort through
- Move clearTimeout from header arrival to stream end, so a server
that stalls after 200 OK headers still hits the timeout
- Make targetPort configurable in proxyConnectTunnel (was hardcoded
443), pass through from _openskyProxyConnect
Golden Ocean Group (GOGL.OL) returns 404 on Yahoo Finance — ticker
delisted or dropped from index. Replace with Eagle Bulk Shipping
(EGLE), same dry bulk carrier sector.
Decodo port 10001 requires TLS. Previous fix removed tls:true override,
causing "Parse Error: Expected HTTP/" on every proxy attempt.
Direct fetch 401 = DODO_API_KEY on Railway is the wrong key. Must use
the REST API key (rVYXvMEjbpQ...), not the Convex component key.
* feat(catalog): seed Dodo prices from Railway relay, Vercel reads only
Dodo API rejects Vercel Edge datacenter IPs (401). Moved price
fetching to ais-relay seed loop on Railway (1h interval, 2h TTL).
Direct fetch first, PROXY_URL fallback if blocked.
Vercel /api/product-catalog now reads from Redis only (gold standard
pattern). Falls back to static prices if Redis empty.
* fix(catalog): change Dodo price seed interval to 12h (TTL 24h)
* fix(catalog): add fixed_price support, restore edge Dodo fallback on purge
P1: Seeder now reads product.price?.price ?? product.price?.fixed_price
(matches previous edge endpoint behavior).
P1: After cache purge, edge endpoint tries Dodo directly as backup
before falling back to static prices. If Dodo succeeds, re-caches.
Prevents 12h gap of fallback-only after manual purge.
* fix(catalog): don't overwrite seeded Redis key from edge, add seed-meta
P1: Edge fallback no longer writes to Redis (avoids overwriting the
Railway-seeded entry with short TTL). Returns result directly with
60s cache.
P2: Seeder now writes seed-meta:product-catalog with fetchedAt,
recordCount, and priceSource for health monitoring.
* fix(catalog): auth proxy, only write dodo-sourced prices, 6h interval
P1: Proxy path now sends Authorization header (ytFetchViaProxy doesn't
support custom headers, so manual CONNECT tunnel with auth).
P1: Only writes to Redis when ALL prices come from Dodo (priceSource=dodo).
Partial/fallback results extend existing TTL but don't overwrite.
Prevents transient outages from pinning stale prices for hours.
Interval: 6h seed, 12h TTL.
* fix(catalog): add health.js monitoring, consistent cachedUntil
P2: Added productCatalog to STANDALONE_KEYS + SEED_META in health.js
(maxStaleMin: 1080 = 3x 6h interval).
P2: Fallback response now includes cachedUntil (consistent contract).
P1: Proxy CONNECT pattern matches existing ytFetchViaProxy (USNI)
which works for Decodo TLS proxies.
* fix(catalog): respect parsed proxy TLS flag instead of forcing tls:true
* fix(relay): proxy fallback for Yahoo/Crypto, isolate OREF proxy, fix Dockerfile
Yahoo Finance and CoinPaprika fail from Railway datacenter IPs (rate
limiting). Added PROXY_URL fallback to fetchYahooChartDirect (used by
5 seeders) and relay chart proxy endpoint. Added shared
_fetchCoinPaprikaTickers with proxy fallback + 5min cache (3 crypto
seeders share one fetch). Added CoinPaprika fallback to CryptoSectors
(previously had none).
Isolated OREF_PROXY_AUTH exclusively for OREF alerts. OpenSky,
seed-military-flights, and _proxy-utils now fall back to PROXY_URL
instead of the expensive IL-exit proxy.
Added seed-climate-news.mjs + _seed-utils.mjs COPY to Dockerfile.relay
(missing since PR #2532). Added pizzint bootstrap hydration to
cache-keys.ts, bootstrap.js, and src/services/pizzint.ts.
* fix(relay): address review — remove unused reverseMap, guard double proxy
- Remove dead reverseMap identity map in CryptoSectors Paprika fallback
- Add _proxied flag to handleYahooChartRequest._tryProxy to prevent
double proxy call on timeout→destroy→error sequence
PizzINT API blocked Vercel datacenter IPs (403), causing the panel to
serve 5h-stale data showing all locations as CLOSED. Moved upstream
fetch to ais-relay seed loop on Railway (10min interval, 30min TTL)
following gold standard pattern. Vercel handler now reads from Redis
seed key only. Also fixed spike_magnitude string bug ("HIGH" instead
of number) from upstream API change.
* feat(news): composite importance score (E1)
Adds a 0-100 importance score to every digest NewsItem, computed from four
weighted signals:
severity x 0.40 (critical=100, high=75, medium=50, low=25, info=0)
source tier x 0.20 (tier 1 wire services score highest)
corroboration x 0.25 (unique sources covering the same story in this cycle)
recency x 0.15 (linear decay to 0 over 30 min)
Key changes:
- proto: add importance_score (9) + corroboration_count (10) to NewsItem
- server/_shared/source-tiers.ts: new shared source tier lookup for server code
- list-feed-digest.ts: corroboration map before truncation, sort by importanceScore
- _classifier.ts: export SEVERITY_VALUES
- notification-relay.cjs: score gate behind IMPORTANCE_SCORE_LIVE env flag;
shadow log to shadow:score-log:v1 (7-day window, always runs)
- ais-relay.cjs: include pubDate + importanceScore in rss_alert payloads
IMPORTANT: IMPORTANCE_SCORE_LIVE=1 must NOT be set until scores are validated
in shadow:score-log:v1 over several days.
* fix(news-alerts): add relay gates, importance score threshold, and RELAY_GATES_READY guard
- ais-relay.cjs: add RELAY_GATES_READY gate that skips tier-4 sources and
stale items (>15min) when relay takes over external notifications
- breaking-news-alerts.ts: add RELAY_GATES_READY constant (VITE_ prefix for
browser) and IMPORTANCE_SCORE_MIN=30 threshold; suppress /api/notify when
relay is active; skip items below importance threshold in selectBest;
propagate importanceScore into BreakingAlert for downstream logging
Fixes gap A (relay tier/recency gate) and gap B (RELAY_GATES_READY guard)
from docs/plans/2026-04-02-003-fix-news-alerts-pr-gaps-plan.md
* fix(relay): remove duplicate shadowLogScore from rebase artifact
Biome noRedeclare: two shadowLogScore definitions landed in
notification-relay.cjs after rebase. Removed the older version
(threshold=65, JSON blob) in favour of the newer E1 version
(SHADOW_SCORE_LOG_KEY constant, timestamped member encoding).
* feat(relay-gates): shadow score log + importance gate + recency filter
* fix(relay-gates): scope score gate to rss_alert only, fix shadow log ordering
P1: score gate fired on all event types; oref_siren, conflict_escalation,
notam_closure etc. have no importanceScore and read as 0, so they were
silently dropped when IMPORTANCE_SCORE_LIVE=1. Gate now only applies to
rss_alert events, which are the only type that carry importanceScore.
P2: shadow log used importanceScore as the sorted-set score, making it
neither time-sortable nor a true rolling window (EXPIRE refresh just
extended lifetime on each write). Now uses timestamp_ms as score for
time ordering, and prunes entries older than 7 days via ZREMRANGEBYSCORE
on each write instead of resetting a TTL. importanceScore is preserved
in the member string for analysis.
* feat(relay-gates): gate browser /api/notify dispatch behind VITE_RELAY_GATES_READY
When set to 'true', the client skips the XHR/fetch call to /api/notify so the
Railway relay becomes the sole notification path, eliminating double-dispatch.
Default (flag absent/false) preserves the existing browser-dispatch path.
* feat(notifications): move event detection server-side in ais-relay
The notification system previously relied on the browser being open to
publish events to wm:events:queue. The relay queue was always empty
unless a user had worldmonitor loaded in a tab.
This moves all alert detection to ais-relay.cjs, which runs 24/7 on
Railway. Five detection points now publish directly to the relay queue:
- OREF sirens: fires on changed && alerts.length > 0 (every 5min)
- News classify: fires when LLM classifies a new title as critical/high (every 15min)
- NWS weather: fires for Extreme/Severe active alerts (every 15min)
- Aviation: fires for closure/severe/major airport disruptions (every 30min)
- NOTAM: fires for newly appeared ICAO airport closures (every 2h)
Infrastructure added: upstashLpush, upstashSetNx, publishNotificationEvent
with 30-min server-side scan-dedup (wm:notif:scan-dedup) to prevent the
same alert firing on every poll cycle.
The browser path (/api/notify) is unchanged and remains as a fallback for
when a tab is open. The relay already handles broadcast events correctly.
* feat(notifications): add market, finance, and threat alert detectors
Extends the server-side notification pipeline with 6 additional detectors:
Market & finance:
- Equity moves: stocks with |change%| >= 5 (high) or >= 10 (critical)
- Commodity moves: oil/gold/etc with |change%| >= 5 (high) or >= 10 (critical)
- Crypto moves: |24h change%| >= 10 (high) or >= 20 (critical)
eventType: market_alert — 1h dedup per symbol/direction
Threat intelligence:
- Cyber threats: new critical/high-severity indicators from Feodo, URLhaus,
OTX, AbuseIPDB — eventType: cyber_threat — 12h dedup per indicator
- Conflict escalation: UCDP events with >= 10 deaths (high) or >= 50 (critical)
eventType: conflict_escalation — 24h dedup per event ID
Logistics/supply chain:
- Corridor risk: shipping corridors with risk score >= 50 (Hormuz, Red Sea, etc.)
eventType: corridor_risk — 1h dedup per corridor
- Shipping stress index: global stress score >= 75 triggers high/critical
eventType: shipping_stress — 2h dedup
Also: publishNotificationEvent gains optional dedupTtl param (default 1800s)
so callers can tune dedup window per event type without touching scan-dedup logic.
All thresholds chosen to avoid alert fatigue while catching genuinely notable
events. Module-level Sets (cyberPrevAlertedIds, ucdpPrevAlertedIds) track
notified items across cycles, auto-cleared at 500 entries.
* fix(notifications): rollback dedup key on LPUSH failure; scope rss_alert dedup per-variant
Two P1 fixes in publishNotificationEvent:
1. If LPUSH fails transiently the scan-dedup key was already set, silently
suppressing retries for the full dedupTtl (30min-24h). Now calls
upstashDel on LPUSH failure so the next poll cycle can retry.
Added upstashDel helper following the same https.request pattern.
2. rss_alert scan-dedup key was global (eventType:title), so the first
variant to classify a headline suppressed all subsequent variants.
A finance or tech user missed alerts if the same headline appeared
earlier in another variant pass. Key now includes variant suffix
so each variant publishes independently; relay delivery dedup handles
duplicates at the consumer side.
* fix(notifications): move rss_alert publish outside seenTitles guard
The publishNotificationEvent call was still inside the seenTitles.has()
check even after scoping the dedup key per-variant. seenTitles is shared
across all classify variants, so once variant A sees a title, variant B
never called publishNotificationEvent at all, bypassing the Redis dedup
entirely. Finance/tech users could still miss alerts for headlines first
encountered in another variant pass.
Fix: separate concerns at the call site. seenTitles guards country
attribution stats only (prevents double-counting). Notification publish
is now unconditional within the classified entry loop, relying solely on
the variant-scoped scan-dedup key in Redis for idempotency.
* fix(notifications): aviation change-detection, NOTAM restart-state, weather title fallback
Aviation P1: Added aviationPrevAlertedSet for in-process change detection so
only airports newly entering severe/major state trigger a notification. The
previous code had no such state, meaning every 30-min poll re-notified for
persistent disruptions since the default dedupTtl matched the poll interval.
dedupTtl raised to 4h to guard against restarts.
NOTAM P1: notamPrevClosed is now persisted to Redis (notam:prev-closed-state:v1)
after each seed and loaded before the first change-detection diff. Previously,
any Railway restart caused the first post-restart poll to treat all currently-
closed airports as new and fire up to 3 false-positive notifications. dedupTtl
raised to 6h (3x the 2h poll interval).
Weather P2: a.headline || a.event can be undefined when neither field is
populated, causing all title-less alerts to collide on a single dedup key.
Falls back to 'Weather alert' so each alert gets an accurate dedup hash.
* fix(macro-signals): direct-first fetch for Yahoo, proxy as fallback only
fetchJsonSafe was proxy-only when proxyAuth was set — it called curlFetch
directly with no direct attempt. This caused Yahoo (and other APIs) to
silently fail whenever the proxy misbehaved, even when the API itself was
healthy. Mirrors the same direct-first pattern already in fredFetchJson.
* fix(shipping): use GOGL.OL (Oslo) ticker for Golden Ocean Group — NASDAQ GOGL returns 404 on Yahoo Finance
* feat(commodity): add gold layer enhancements from fork review
Enrich the commodity variant with learnings from Yazan-Abuawwad/gold-monitor fork:
- Add 10 missing gold mines to MINING_SITES: Muruntau (world's largest
open-pit gold mine), Kibali (DRC), Sukhoi Log (Russia, development),
Ahafo (Ghana), Loulo-Gounkoto (Mali), South Deep (SA), Kumtor
(Kyrgyzstan), Yanacocha (Peru), Cerro Negro (Argentina), Tropicana
(Australia). Covers ~40% of top-20 global mines previously absent.
- Add XAUUSD=X spot gold and 9 FX pairs (EUR, GBP, JPY, CNY, INR, AUD,
CHF, CAD, TRY) to shared/commodities.json. All =X symbols auto-seeded
via existing seedCommodityQuotes() — no new seeder needed. Registered
in YAHOO_ONLY_SYMBOLS in both _shared.ts and ais-relay.cjs.
- Add XAU/FX tab to CommoditiesPanel showing gold priced in 10 currencies.
Computed live from GC=F * FX rates. Commodity variant only.
- Fix InsightsPanel brief title: commodity variant now shows
"⛏️ COMMODITY BRIEF" instead of "🌍 WORLD BRIEF".
- Route commodity variant daily market brief to commodity feed categories
(commodity-news, gold-silver, mining-news, energy, critical-minerals)
via new newsCategories option on BuildDailyMarketBriefOptions.
- Add Gold Silver Worlds + FX Empire Gold direct RSS feeds to gold-silver
panel (9 sources total, up from 7).
* fix(commodity): address review findings from PR #2464
- Fix USDCHF=X multiply direction: was true (wrong), now false (USD/CHF is USD-per-CHF convention)
- Fix newsCategories augments BRIEF_NEWS_CATEGORIES instead of replacing (preserves macro/Fed context in commodity brief)
- Add goldsilverworlds.com + www.fxempire.com to RSS allowlist (api + shared + scripts/shared)
- Rename "Metals" tab label conditionally: commodity variant gets "Metals", others keep "Commodities"
- Reset _tab to "commodities" when hasXau becomes false (prevent stale XAU tab re-activation)
- Add Number.isFinite() guard in _renderXau() before computing xauPrice
- Narrow fxMap filter to =X symbols only
- Collapse redundant two-branch number formatter to Math.round().toLocaleString()
- Remove XAUUSD=X from shared/commodities.json: seeded but never displayed (saves 150ms/cycle)
* feat(mcp): add get_commodity_geo tool and update get_market_data description
* fix(commodity): correct USDCHF direction, replace headline categories, restore dep overrides
* fix(commodity): empty XAU grid fallback and restore FRED timeout to 20s
* fix(commodity): remove XAU/USD from MCP description, revert Metals tab label
* fix(commodity): remove dead XAUUSD=X from YAHOO_ONLY_SYMBOLS
XAU widget uses GC=F as base price, not XAUUSD=X. Symbol was never
seeded (not in commodities.json) and never referenced in the UI.
Map projections can send lamin/lamax outside [-90,90] (e.g. a zoomed-out
viewport). The isFinite validation passes these through, but Wingbits rejects
la < -90 with a Zod too_small error (HTTP 400). Clamp all four bbox values
to valid lat/lon ranges before computing centerLat/centerLon so the API
payload is always valid regardless of what the client sends.
* feat(aviation): add SearchGoogleFlights and SearchGoogleDates RPCs
Port Google Flights internal API from fli Python library as two new
aviation service RPCs routed through the Railway relay.
- Proto: SearchGoogleFlights and SearchGoogleDates messages and RPCs
- Relay: handleGoogleFlightsSearch and handleGoogleFlightsDates handlers
with JSONP parsing, 61-day chunking for date ranges, cabin/stops/sort mappers
- Server handlers: forward params to relay google-flights endpoints
- gateway.ts: no-store for flights, medium cache for dates
* feat(mcp): expose search_flights and search_flight_prices_by_date tools
* test(mcp): update tool count to 24 after adding search_flights and search_flight_prices_by_date
* fix(aviation): address PR review issues in Google Flights RPCs
P1: airline filtering — use gfParseAirlines() in relay (handles comma-joined
string from codegen) and parseStringArray() in server handlers
P1: partial chunk failure now sets degraded: true instead of silently
returning incomplete data as success; relay includes partial: true flag
P2: round-trip date search validates trip_duration > 0 before proceeding;
returns 400 when is_round_trip=true and duration is absent/zero
P2: relay mappers accept user-friendly aliases ('0'/'1' for max_stops,
'price'/'departure' for sort_by) alongside symbolic enum values;
MCP tool docs updated to match
* fix(aviation): use cachedFetchJson in searchGoogleDates for stampede protection
Medium cache tier (10 min) requires Redis-level coalescing to prevent
concurrent requests from all hitting the relay before cache warms.
Cache key includes all request params (sorted airlines for stable keys).
* fix(aviation): always use getAll() for airlines in relay; add multi-airline tests
The OR short-circuit (get() || getAll()) meant get() returned the first
airline value (truthy), so getAll() never ran and only one airline was
forwarded to Google. Fix: unconditionally use getAll().
Tests cover: multi-airline repeated params, single airline, empty array,
comma-joined string from codegen, partial degraded flag propagation.
Previously each seeder (ais-relay.cjs, _seed-utils.mjs, seed-fear-greed.mjs,
seed-disease-outbreaks.mjs) had its own inline resolveProxy() with slightly
different implementations. This caused USNI seeding to fail because
parseProxyUrl() only handled URL format while PROXY_URL uses Decodo
host:port:user:pass format.
- Add scripts/_proxy-utils.cjs with parseProxyConfig(), resolveProxyConfig(),
resolveProxyString() handling both http://user:pass@host:port and
host:port:user:pass formats
- ais-relay.cjs: require _proxy-utils.cjs, alias parseProxyUrl = parseProxyConfig
- _seed-utils.mjs: import resolveProxyString via createRequire, delegate resolveProxy()
- seed-fear-greed.mjs, seed-disease-outbreaks.mjs: remove inline resolveProxy(),
import from _seed-utils.mjs instead
* fix(widgets): restore iframe content after drag, remove color-cycle button
- Fix drag-induced blank content: use WeakMap keyed by iframe element to persist
HTML across DOM moves; persistent load listener (no {once}) re-posts on every
browser re-navigation triggered by drag/drop repositioning
- Remove cycleAccentColor, ACCENT_COLORS, and colorBtn from CustomWidgetPanel
header; chatBtn (sparkle) and PRO badge remain; applyAccentColor kept for
saved specs
- Update tests: remove ACCENT_COLORS count test, saveWidget persistence test,
and changeAccent i18n assertion (all for deleted feature)
* fix(widgets): use correct postMessage key 'html' not 'storedHtml'
* fix(widgets): remove duplicate panel title header, fix sandbox CSP beacon error
- System prompt: NEVER add .panel-header or title to widget body; outer panel
frame already shows the title; updated both basic and PRO prompts
- widget-sanitizer: strip leading .panel-header from generated HTML as safety
net in both wrapWidgetHtml and wrapProWidgetHtml
- vercel.json: add https://static.cloudflareinsights.com to sandbox script-src
so Cloudflare beacon injection no longer triggers CSP console errors
* fix(widgets): correct iframe font by anchoring html+body font-family with !important
Prevent users from hijacking the widget agent for off-topic AI tasks
or prompt injection attacks that would waste Anthropic API budget.
Layer 1 -- Hard reject before any API call (isWidgetInjectionAttempt):
- Detects instruction override patterns (ignore/disregard/forget previous
instructions), role hijacking (act as, pretend to be, your new persona),
prompt exfiltration (show/reveal system prompt), structural markers
([SYSTEM], <system>, ###system), and jailbreak vocabulary (DAN, developer
mode). Returns 400 with no tokens spent.
Layer 2 -- System prompt scope enforcement (both basic and PRO):
- Added NON-NEGOTIABLE refusal block at top of both system prompts.
Model is instructed to refuse all off-topic requests silently with a
fixed widget-html stub response instead of engaging or explaining.
Layer 3 -- Tool result sanitization (sanitizeToolContent):
- Applied to all web search results and WM API responses before they
enter the conversation context. Strips obvious injection patterns from
untrusted third-party content (snippets, titles, API fields).
* chore: redeploy to pick up WORLDMONITOR_VALID_KEYS fix
* feat(widget-agent): bootstrap catalog + tool budget + emergency emit
Fix design drift and tool loop exhaustion in the widget agent.
Bootstrap data source:
- Add /api/bootstrap?keys=<key> as the PREFERRED data source in both
system prompts. Bootstrap is pre-seeded, instant, and returns the
exact same data the dashboard panels display (techReadiness, riskScores,
marketQuotes, earthquakes, wildfires, etc. -- 60+ keys).
- Reorganize prompt: Option 1 = bootstrap (grouped by category), Option 2
= live RPCs (only for parameterized/custom queries). search_web is last resort.
Tool budget (Option E):
- Add "max 3 tool calls" rule to both system prompts. After 2 failed calls,
agent must emit HTML immediately -- no more probing. Prevents the agent
burning all turns on sequential endpoint guesses.
Emergency emit (Option D):
- On turn maxTurns-2, inject a FINAL TURN directive and pass tools:[]
to force end_turn instead of another tool_use. Agent must emit HTML
now with whatever data it collected.
Partial recovery:
- On loop exhaustion, scan all assistant messages for any partial
widget-html markers and emit the best one found, instead of always
returning an error to the user.
* fix(widget): allow cdn.jsdelivr.net in sandbox CSP + Sentry error tracking
- Fix Chart.js source map noise: relax sandbox connect-src from 'none' to
https://cdn.jsdelivr.net (both vercel.json header and meta CSP in buildWidgetDoc)
- Add Sentry API 5xx capture in premiumFetch via reportServerError() -- fires on
any status >= 500 before response is returned, tags kind: api_5xx
- Add securitypolicyviolation listener in main.ts for parent-page CSP violations,
filters browser-extension and blob origins, tags kind: csp_violation
* feat(widget): inject panel design system into PRO widget sandbox
Problem: PRO widgets used a disconnected design (large bold titles,
custom tab buttons, hardcoded hex colors) because the sandbox iframe
had no panel CSS classes and the agent had no examples to follow.
Fix:
- buildWidgetDoc: add .panel-header, .panel-title, .panel-tabs,
.panel-tab, .panel-tab.active, .disp-stats-grid, .disp-stat-box,
.disp-stat-value, .disp-stat-label, and --accent CSS variable to
the iframe's <style> block so they work without a custom <style>
- WIDGET_PRO_SYSTEM_PROMPT: add concrete HTML examples for panel
header+tabs, stat boxes, and Chart.js color setup using CSS vars;
prohibit h1/h2/h3 large titles; document the switchTab() pattern
- Test: assert all panel classes and --accent are present in document
Agent now has classes to USE instead of inventing its own styling.
* feat(widget-agent): open API allowlist to all /api/ paths with compact taxonomy
Problem: widget agent only knew 14 hardcoded endpoints and prioritized
search_web even when a WorldMonitor data source was available.
- Replace WIDGET_ALLOWED_ENDPOINTS Set with isWidgetEndpointAllowed()
function: permits any /api/ path, blocks inference/write endpoints
(analyze-stock, backtest-stock, summarize-article, classify-event, etc.)
- Replace per-URL endpoint lists in both WIDGET_SYSTEM_PROMPT and
WIDGET_PRO_SYSTEM_PROMPT with a compact service-grouped taxonomy:
service + method names only, no full URL repeated 60 times (~400
tokens vs ~1200 for 4x more endpoint coverage)
- Strengthen prioritization: "ALWAYS use first, ONLY fall back to
search_web if no matching service exists" (was "preferred for topics")
- Add 30+ new endpoints: earthquakes, wildfires, cyber threats, sanctions,
consumer prices, FRED series, BLS, Big Mac, fuel, grocery, ETF flows,
shipping rates, chokepoints, critical minerals, GPS interference, etc.
* fix(csp): add safari-web-extension: scheme to CSP violation filter
* fix(usni): try US proxy fallback when IL proxy fails for USNI fleet scrape
IL-targeted OREF proxy was getting Cloudflare-blocked on USNI (US news site).
Now tries IL proxy first, then YOUTUBE_PROXY_URL (US Decodo proxy), then direct.
Prevents STALE_SEED health warnings when IL exit nodes are blocked.
* fix(relay): use PROXY_URL for USNI, not OREF_PROXY_AUTH (IL-only)
* fix(usni): hoist PROXY_URL constant, separate JSON parse errors from proxy errors
* feat(intelligence): emit news:threat:summary:v1 from relay classify loop for CII
During seedClassifyForVariant(), attribute each title to ISO2 countries
while both title and classification result are in scope. At the end of
seedClassify(), merge per-country threat counts across all variants and
write news:threat:summary:v1 (20min TTL) with { byCountry: { [iso2]: {
critical, high, medium, low, info } }, generatedAt }.
get-risk-scores.ts reads the new key via fetchAuxiliarySources() and
applies weighted scores (critical→4, high→2, medium→1, low→0.5, info→0,
capped at 20) per country into the information component of CII eventScore.
Closes#2053
* fix(intelligence): register news:threat-summary in health.js and expand tests
- Add newsThreatSummary to BOOTSTRAP_KEYS (seed-meta:news:threat-summary,
maxStaleMin: 60) so relay classify outages surface in health dashboard
- Add 4 tests: boost verification, cap-at-20, unknown-country safety,
null-threatSummary zero baseline
* fix(classify): de-dup cross-variant titles and attribute to last-mentioned country
P1-A: seedClassify() was summing byCountry across all 5 variants (full/tech/
finance/happy/commodity) without de-duplicating. Shared feeds (CNBC, Yahoo
Finance, FT, HN, Ars) let a single headline count up to 4x before reaching
CII, saturating threatSummaryScore on one story.
Fix: pass seenTitles Set into seedClassifyForVariant; skip attribution for
titles already counted by an earlier variant.
P1-B: matchCountryNamesInText() was attributing every country mentioned in a
headline equally. "UK and US launch strikes on Yemen" raised GB, US, and YE
with identical weight, inflating actor-country CII.
Fix: return only the last country in document order — the grammatical object
of the headline, which is the primary affected country in SVO structure.
* fix(classify): replace last-position heuristic with preposition-pattern attribution
The previous "last-mentioned country" fix still failed for:
- "Yemen says UK and US strikes hit Hodeidah" → returned US (wrong)
- "US strikes on Yemen condemned by Iran" → returned IR (wrong)
Both failures stem from position not conveying grammatical role. Switch to a
preposition/verb-pattern approach: only attribute to a country that immediately
follows a locative preposition (in/on/against/at/into/targeting/toward) or an
attack verb (invades/attacks/bombs/hits/strikes). No pattern match → return []
(skip attribution rather than attribute to the wrong country).
* fix(classify): fix regex hitting, gaza/hamas geo mapping, seed-meta always written
- hitt?(?:ing|s)? instead of hit(?:s|ting)? so "hitting" is matched
- gaza → PS (Palestinian Territories), hamas → PS (was IL)
- seed-meta:news:threat-summary written unconditionally so health check
does not fire false alerts during no-attribution runs
Clerk PRO users had no path to authenticate to the widget agent.
Their PRO status comes from getAuthState().user?.role === 'pro',
not from wm-widget-key/wm-pro-key (tester keys), so getWidgetAgentKey()
returned '' and every request got 403.
Fix:
- Add api/widget-agent.ts (Vercel edge): validates Clerk Bearer JWT
(plan === 'pro') OR tester keys, then proxies SSE to relay with
real server-side WIDGET_AGENT_KEY/PRO_WIDGET_KEY. Clerk users never
see the relay secrets.
- Update widgetAgentUrl() to route prod browser traffic through
/api/widget-agent. runtime.ts interceptor injects Authorization:
Bearer <clerk_jwt> automatically for same-origin requests.
- Desktop still bypasses to relay directly (SSE streaming compat).
Also fix two relay issues from code review of #2304:
- P2: requireWidgetAgentAccess returned 503 when WIDGET_AGENT_KEY was
empty even if PRO_WIDGET_KEY was set — blocks PRO-only deployments.
Fix: guard is now !widgetKeyConfigured && !proKeyConfigured.
- P1: request admitted via pro key with no body.tier defaulted to
basic rate-limit bucket. requireWidgetAgentAccess now returns
admittedAs:'pro'|'basic'; handleWidgetAgentRequest uses it to
default tier to 'pro' and skip the redundant pro-key re-check.
PRO users who have wm-pro-key but no wm-widget-key were getting 403
because requireWidgetAgentAccess rejected the empty X-Widget-Key before
reaching the PRO key check. Allow a valid X-Pro-Key to satisfy the gate
on its own since PRO tier is a superset of basic.
* fix(wingbits): cap bbox dimensions to 2000 nm to stop persistent 400 errors
The v1 API rejects requests with overly large bounding boxes. Three call
sites were sending sizes that exceeded the API limit:
- Global callsign fallback: replaced w:21600 h:10800 full-globe box with
three 2000 nm regional areas (Europe/Middle East, Asia-Pacific, Americas)
- Viewport handler: added Math.min(..., WINGBITS_MAX_BOX_NM) cap so
zoomed-out views don't produce multi-thousand-nm requests
- Theater posture handler: same cap on per-theater w/h computation
Also added response body logging to both error paths so the actual API
error message surfaces in logs rather than just the status code.
* fix(wingbits): log error body on regional callsign fallback non-OK response
Root cause: seedWeatherAlerts() had two early-return paths that skipped the
seed-meta upstash write — alerts.length===0 (quiet weather) and !resp.ok.
After a transient NWS fetch failure, subsequent successful-but-empty runs
never bumped fetchedAt, causing health.js to see the old timestamp grow stale.
- Update seed-meta on alerts.length===0 (NWS OK, just no active alerts)
- Keep !resp.ok path as-is (prolonged NWS outage should still alert)
- Add weatherAlerts to EMPTY_DATA_OK_KEYS (0 alerts = valid quiet state)
Russell 2000 (^RUT):
- Add to MARKET_SYMBOLS and YAHOO_ONLY in ais-relay.cjs
- Covers the 4th major US index missing from the panel
GSCPI (NY Fed Global Supply Chain Pressure Index):
- New seedGscpi() loop in ais-relay.cjs — fetches monthly CSV from
newyorkfed.org (no API key required), parses wide-format vintage CSV
- Stored as economic:fred:v1:GSCPI:0 in FRED-compatible format so the
existing GetFredSeriesBatch RPC serves it without any proto changes
- TTL 72h (3x 24h interval), retry 20min on failure — gold standard pattern
- Add GSCPI to ALLOWED_SERIES in get-fred-series-batch.ts
- Add gscpi to STANDALONE_KEYS + SEED_META in health.js (maxStaleMin 2880)
* feat(commodities): expand tracking to cover agricultural and coal futures
Adds 9 new commodity symbols to cover the price rally visible in our
intelligence feeds: Newcastle Coal (MTF=F), Wheat (ZW=F), Corn (ZC=F),
Soybeans (ZS=F), Rough Rice (ZR=F), Coffee (KC=F), Sugar (SB=F),
Cocoa (CC=F), and Cotton (CT=F).
Also fixes ais-relay seeder to use display names from commodities.json
instead of raw symbols, so seeded data is self-consistent.
* fix(commodities): gold standard cache, 3-col grid, cleanup
- Add upstashExpire on zero-quotes failure path so bootstrap key TTL
extends during Yahoo outages (gold standard pattern)
- Remove unreachable fallback in retry loop (COMMODITY_META always has
the symbol since it mirrors COMMODITY_SYMBOLS)
- Switch commodities panel to 3-column grid (19 items → ~7 rows vs 10)
* fix(relay): add global Wingbits fallback for callsign-only index misses
When a callsign search hits the relay but the flight isn't in any
recent viewport (index miss), the relay now attempts a worldwide
Wingbits API call filtered by callsign server-side.
Previously: index miss → immediate empty response
Now: index miss → global bbox call → filter by callsign → return matches
Fallback is graceful: timeouts/errors return empty rather than 5xx.
Matched flights are also written into the index for subsequent queries.
* fix(aviation): try Wingbits before OpenSky for callsign-only searches
Commercial flights like UAE20, THY6260 are Wingbits-exclusive and
invisible to OpenSky receivers. The old order (OpenSky first) wasted
10s+ on OpenSky global states/all (rate-limited, no callsign filter),
then returned early if any OpenSky positions existed — never reaching
Wingbits. Result: source:"none" even when the flight was visible on map.
New order for callsign-only queries:
1. Wingbits relay (index + global fallback on miss) — fast and exact
2. OpenSky skipped (no callsign filter, rate-limited, wrong source)
OpenSky and Wingbits bbox fallback paths unchanged for bbox/icao24 queries.
The global Wingbits bbox call (-80/-180/80/180) was unreliable — Wingbits
returns 401/empty for worldwide queries. Replace it with an in-memory index
populated from every successful bbox response. Callsign-only queries check the
index (fresh within 5 min) without hitting the Wingbits API.
Also fix cache key bug: callsign-only searches previously fell through to
'aviation:track:all:v1' (shared across all searches). Now keyed per callsign.
Use shorter TTLs: 60s positive, 10s negative for callsign searches.
Result: if CTN465 or SWR785B is visible in any user's viewport, the relay
index has it and CMD+K 'flight CTN465' returns it immediately.
- Extend /wingbits/track relay endpoint to accept ?callsign=EK36 without bbox,
using global bounding box (-80/-180/80/180) for worldwide aircraft lookup
- Filter by callsign in relay response loop to return only matching flights
- Server: try Wingbits for callsign-only searches (previously only tried for bbox)
- Server: return empty positions (not simulated) for explicit callsign/icao24 lookups
- Client: filter out simulated-source positions before displaying search results
* fix(panels): hide finance-only panels on non-finance variants
stock-analysis, stock-backtest, and daily-market-brief are FINANCE_PANELS-only
but persisted panelSettings from a previous finance/full session caused them
to appear on tech variant. Guard creation with variantPanelKeys set derived
from VARIANT_DEFAULTS[SITE_VARIANT].
* fix(pro): remove duplicate Create with AI button, gate MCP connect as pro
With unified isProUser() both widget creation buttons showed simultaneously.
Remove the basic-tier 'Create with AI' block (duplicate of 'Create Interactive
Widget'). Gate 'Connect with MCP' with isProUser() — was previously ungated.
* feat(pro): gate export (⬇) and playback (⏪) toolbar buttons for pro users
Both setupExportPanel() and setupPlaybackControl() now return early
unless isProUser() — either wm-widget-key or wm-pro-key grants access.
* revert(panels): remove incorrect variant guard on stock/daily-market panels
The variantPanelKeys.has() guards were wrong — shouldCreatePanel() already
handles variant defaults via panelSettings. Users who manually enable these
panels on any variant should see them (locked if not pro, unlocked if pro).
The guard broke cross-variant user customization.
* fix(resilience): extend TTLs and add Finnhub/FRED fallbacks for macro signals
- MACRO_TTL 1800s → 21600s (6h) so stale data serves during Yahoo outages
- MARKET_SEED_TTL 1800s → 7200s (2h) in ais-relay for sectors/quotes
- Finnhub stock/candle fallback for QQQ and XLP when Yahoo returns null
- Finnhub crypto/candle fallback for BTC (BINANCE:BTCUSDT) when Yahoo fails
- FRED DEXJPUS fallback for JPY/USD historical prices (no new API key needed)
Prevents Macro Stress and Sector Heatmap panels from showing "unavailable"
during Yahoo Finance 429 rate-limit windows from Railway IPs.
