mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
main
507 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Elie Habib
|
79ec6e601b | feat(prefs): Phase 0 — sync primitives and notification scaffolding (#2503) | ||
|
Elie Habib
|
f482c13d6b |
fix(gateway): gate CDN-Cache-Control on trusted origin (#2496)
* fix(gateway): gate CDN-Cache-Control on trusted origin to close cache bypass No-origin server-side requests (external scrapers) no longer receive a CDN-Cache-Control header, so Vercel CDN never caches their responses. Every request without a trusted origin must reach the edge function, ensuring validateApiKey() always runs and returns 401 for unauthenticated callers. Trusted origins (worldmonitor.app, Vercel previews, Tauri) are unchanged — CDN caching behaviour and hit rates for the app itself are unaffected. * docs: external API usage learnings from aeris-AgentR and HOPEFX * docs: fix markdown lint (blanks around lists) * docs: move external-learnings to separate PR |
||
|
Elie Habib
|
aa3e84f0ab |
feat(economic): Economic Stress Composite Index panel (FRED 6-series, 0-100 score) (#2461)
* feat(economic): Economic Stress Composite Index panel (FRED 6-series, 0-100 score) - Add T10Y3M and STLFSI4 to FRED_SERIES in seed-economy.mjs and ALLOWED_SERIES - Create proto message GetEconomicStressResponse with EconomicStressComponent - Register GetEconomicStress RPC in EconomicService (GET /api/economic/v1/get-economic-stress) - Add seed-economic-stress.mjs: reads 6 pre-seeded FRED keys via Redis pipeline, computes weighted composite score (0-100) with labels Low/Moderate/Elevated/Severe/Critical - Create server handler get-economic-stress.ts reading from economic:stress-index:v1 - Register economicStress in BOOTSTRAP_CACHE_KEYS (both cache-keys.ts and api/bootstrap.js) as slow tier - Add gateway.ts cache tier entry (slow) for new RPC route - Create EconomicStressPanel.ts: composite score header, gradient needle bar, 2x3 component grid with score bars, desktop notification on threshold cross (>=70, >=85) - Wire economic-stress panel in panels.ts (all 4 variants), panel-layout.ts, and data-loader.ts - Regenerate OpenAPI docs and TypeScript client/server types * fix(economic-stress): null for missing FRED data + tech variant panel - Add 'economic-stress' panel to TECH_PANELS defaults (was missing, only appeared in full/finance/commodity variants) - Seed: write rawValue: null + missing: true when no valid FRED observation found, preventing zero-valued yield curve/bank spread readings from being conflated with missing data - Proto: add missing bool field to EconomicStressComponent message; regenerate client/server types + OpenAPI docs - Server handler: propagate missing flag from Redis; pass rawValue: 0 on wire when missing to satisfy proto double type - Panel: guard on c.missing (not rawValue === 0) to show grey N/A card with no score bar for unavailable components * fix(economic-stress): add purple Critical zone to gradient bar Update gradient stops to match the 5 equal tier boundaries (0-20-40-60-80-100), adding the #8e44ad purple stop at 80% so scores 80-100 render as Critical purple instead of plain red. |
||
|
Elie Habib
|
ba54dc12d7 |
feat(commodity): gold layer enhancements (#2464)
* feat(commodity): add gold layer enhancements from fork review Enrich the commodity variant with learnings from Yazan-Abuawwad/gold-monitor fork: - Add 10 missing gold mines to MINING_SITES: Muruntau (world's largest open-pit gold mine), Kibali (DRC), Sukhoi Log (Russia, development), Ahafo (Ghana), Loulo-Gounkoto (Mali), South Deep (SA), Kumtor (Kyrgyzstan), Yanacocha (Peru), Cerro Negro (Argentina), Tropicana (Australia). Covers ~40% of top-20 global mines previously absent. - Add XAUUSD=X spot gold and 9 FX pairs (EUR, GBP, JPY, CNY, INR, AUD, CHF, CAD, TRY) to shared/commodities.json. All =X symbols auto-seeded via existing seedCommodityQuotes() — no new seeder needed. Registered in YAHOO_ONLY_SYMBOLS in both _shared.ts and ais-relay.cjs. - Add XAU/FX tab to CommoditiesPanel showing gold priced in 10 currencies. Computed live from GC=F * FX rates. Commodity variant only. - Fix InsightsPanel brief title: commodity variant now shows "⛏️ COMMODITY BRIEF" instead of "🌍 WORLD BRIEF". - Route commodity variant daily market brief to commodity feed categories (commodity-news, gold-silver, mining-news, energy, critical-minerals) via new newsCategories option on BuildDailyMarketBriefOptions. - Add Gold Silver Worlds + FX Empire Gold direct RSS feeds to gold-silver panel (9 sources total, up from 7). * fix(commodity): address review findings from PR #2464 - Fix USDCHF=X multiply direction: was true (wrong), now false (USD/CHF is USD-per-CHF convention) - Fix newsCategories augments BRIEF_NEWS_CATEGORIES instead of replacing (preserves macro/Fed context in commodity brief) - Add goldsilverworlds.com + www.fxempire.com to RSS allowlist (api + shared + scripts/shared) - Rename "Metals" tab label conditionally: commodity variant gets "Metals", others keep "Commodities" - Reset _tab to "commodities" when hasXau becomes false (prevent stale XAU tab re-activation) - Add Number.isFinite() guard in _renderXau() before computing xauPrice - Narrow fxMap filter to =X symbols only - Collapse redundant two-branch number formatter to Math.round().toLocaleString() - Remove XAUUSD=X from shared/commodities.json: seeded but never displayed (saves 150ms/cycle) * feat(mcp): add get_commodity_geo tool and update get_market_data description * fix(commodity): correct USDCHF direction, replace headline categories, restore dep overrides * fix(commodity): empty XAU grid fallback and restore FRED timeout to 20s * fix(commodity): remove XAU/USD from MCP description, revert Metals tab label * fix(commodity): remove dead XAUUSD=X from YAHOO_ONLY_SYMBOLS XAU widget uses GC=F as base price, not XAUUSD=X. Symbol was never seeded (not in commodities.json) and never referenced in the UI. |
||
|
Elie Habib
|
4435d43436 |
feat(analyst): WM Analyst — pro chat panel with streaming SSE (#2459)
* feat(analyst): add WM Analyst pro chat panel with streaming SSE Pro-gated conversational AI panel that assembles live context from 9 Redis sources (news insights, risk scores, market implications, forecasts, macro signals, prediction markets, stock/commodity quotes, country brief) and streams LLM responses token-by-token via SSE. - api/chat-analyst.ts: standalone Vercel edge function with isCallerPremium auth gate, history trimming, and text/event-stream SSE response - server/.../chat-analyst-context.ts: parallel Redis assembly via Promise.allSettled with graceful degradation on partial failure - server/.../chat-analyst-prompt.ts: domain-focused system prompt builder with geo/market/military/economic emphasis modes - server/_shared/llm.ts: callLlmReasoningStream() streaming variant returning ReadableStream emitting delta/done/error SSE events - src/components/ChatAnalystPanel.ts: Panel subclass with domain chips, quick actions, streaming indicator, export, and AbortController cleanup - Wired into WEB_PREMIUM_PANELS + lazyPanel in panel-layout.ts Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * chore: gitignore docs/plans and docs/brainstorms (local planning artifacts) * refactor(analyst): rename panel from SIGINT Analyst to WM Analyst * fix(chat-analyst): address security and correctness review findings - Add VALID_DOMAINS allowlist for domainFocus to prevent prompt injection - Sanitize history message content with sanitizeForPrompt() - Reduce LLM timeout to 25s to stay within Vercel Edge wall-clock limit - Surface degraded context as SSE event when >4 Redis sources fail - Remove unused _domainFocus param from assembleAnalystContext - Remove redundant .catch(() => null) inside Promise.allSettled calls - Fix callLlmReasoningStream return type (always returns stream, not null) - Remove dead sseError function (unreachable null-stream path) - Add chat-analyst to apiKeyPanels so API key holders can access the panel - Use requestAnimationFrame for scroll-to-bottom to prevent layout thrash - Handle degraded SSE event in client with inline notice - Fix orphaned history.push in !res.ok error path - Use pushHistory() in incomplete stream path to avoid duplication * feat(chat-analyst): GDELT headlines, domain filtering, source chips, preset UX - Add GDELT live headlines as 10th context source (domain-aware topic, 8s timeout, graceful fallback, parallel with Redis via Promise.allSettled) - Add activeSources[] to AnalystContext — labels for non-empty data fields - Restore domainFocus param in assembleAnalystContext for GDELT topic mapping - Add DOMAIN_SECTIONS map: only inject relevant context sections per domain (market skips risk/world brief, military skips market sections, etc.) - Replace always-inject-all approach — smaller prompts for focused queries - Raise word limit 250→350 words; add bold headers + cite figures instruction - Replace conditional degraded SSE event with always-present meta event: { meta: { sources: string[], degraded: bool } } as first SSE event - Upgrade QUICK_ACTIONS to { label, icon, query } objects; chips show icon + short label instead of raw query text - Add renderSourceChips(): source badge row above each analyst response, rendered from meta.sources before first token arrives * fix(chat-analyst): address P1/P2 review findings - Probability threshold: use prob > 1 (not prob * 100 > 1) so fractions like 0.75 render as 75% not 1% - CHROME_UA: import from server/_shared/constants instead of local Chrome/124 - GDELT timeout: reduce 8s to 2.5s to avoid breaching Vercel edge stream limit - GDELT headlines: use sanitizeForPrompt instead of sanitizeHeadline (GDELT is an external unauthenticated source, deserves full sanitization) - Rate limiting: add checkRateLimit after isPremium gate to bound LLM cost - LlmStreamOptions: omit provider field (silently ignored by callLlmReasoningStream) - ChatAnalystPanel: append text nodes during streaming, render markdown only on finalize to eliminate per-token layout recalculations - geoContext: document as Phase 2 / agent-accessible pending map wiring * test(chat-analyst): add 18 tests for buildAnalystSystemPrompt domain filtering and config alignment * fix(llm-stream): abort propagation and mid-stream provider splice P1 — callLlmReasoningStream now has a cancel() handler that sets a streamClosed flag and aborts the active provider fetch via AbortController. Accepts an optional signal (req.signal from the edge handler) wired to the per-fetch controller so client disconnect cancels upstream work. prependSseEvent propagates cancel() to its inner reader. P2 — hasContent is now declared outside the try block so the catch path can see it. If a provider has already emitted delta chunks and then throws (mid-stream network error), the stream closes with {done:true} rather than falling through to the next provider and splicing a second answer. * fix(llm-stream): timeout scope and truncated-response signaling Timeout regression: clearTimeout was firing immediately after fetch() returned headers, so timeoutMs no longer bounded the streaming body read. Moved it to after the while loop (normal completion) and to the !resp.ok early-exit path; catch still clears it on throw. Partial-success misreport: when a provider threw mid-stream after emitting deltas, the catch emitted {done:true} which the client treated as clean completion and committed the truncated answer to history. Now the stream closes without a done event so readStream() returns 'incomplete'. The panel appends a visible truncation note and skips pushHistory to avoid poisoning the conversation context with a partial answer. * fix(chat-analyst): source chip CSS and italic markdown rendering Add .chat-source-chips / .chat-source-chip styles that were missing — source labels (Brief, Risk, Forecasts, etc.) were rendering as unstyled run-on text. Added .chat-source-chip--warn for the degraded indicator. Add italic support to basicMarkdownToHtml() so *...* renders as <em> instead of literal asterisks. Bold is processed first to avoid partial matches on **...**. Fixes the "Response may be incomplete" and "Response cut off" truncation messages which used italic markers. * fix(llm-stream): [DONE] sentinel now exits the outer read loop break inside the inner for-lines loop only exited that loop; the outer while continued reading until the TCP connection closed naturally. Added providerDone flag so [DONE] breaks both loops immediately. --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Elie Habib
|
d01469ba9c |
feat(aviation): add SearchGoogleFlights and SearchGoogleDates RPCs (#2446)
* feat(aviation): add SearchGoogleFlights and SearchGoogleDates RPCs
Port Google Flights internal API from fli Python library as two new
aviation service RPCs routed through the Railway relay.
- Proto: SearchGoogleFlights and SearchGoogleDates messages and RPCs
- Relay: handleGoogleFlightsSearch and handleGoogleFlightsDates handlers
with JSONP parsing, 61-day chunking for date ranges, cabin/stops/sort mappers
- Server handlers: forward params to relay google-flights endpoints
- gateway.ts: no-store for flights, medium cache for dates
* feat(mcp): expose search_flights and search_flight_prices_by_date tools
* test(mcp): update tool count to 24 after adding search_flights and search_flight_prices_by_date
* fix(aviation): address PR review issues in Google Flights RPCs
P1: airline filtering — use gfParseAirlines() in relay (handles comma-joined
string from codegen) and parseStringArray() in server handlers
P1: partial chunk failure now sets degraded: true instead of silently
returning incomplete data as success; relay includes partial: true flag
P2: round-trip date search validates trip_duration > 0 before proceeding;
returns 400 when is_round_trip=true and duration is absent/zero
P2: relay mappers accept user-friendly aliases ('0'/'1' for max_stops,
'price'/'departure' for sort_by) alongside symbolic enum values;
MCP tool docs updated to match
* fix(aviation): use cachedFetchJson in searchGoogleDates for stampede protection
Medium cache tier (10 min) requires Redis-level coalescing to prevent
concurrent requests from all hitting the relay before cache warms.
Cache key includes all request params (sorted airlines for stable keys).
* fix(aviation): always use getAll() for airlines in relay; add multi-airline tests
The OR short-circuit (get() || getAll()) meant get() returned the first
airline value (truthy), so getAll() never ran and only one airline was
forwarded to Google. Fix: unconditionally use getAll().
Tests cover: multi-airline repeated params, single airline, empty array,
comma-joined string from codegen, partial degraded flag propagation.
|
||
|
Elie Habib
|
e43292b057 |
feat(market-implications): add transmission chain to implication cards (#2439)
Models the causal path from geopolitical event to price impact as a 2-4 node chain on each card. Each node has a label, impact_type, and logic sentence. Nodes are tap/click-expandable inline. - Proto: add TransmissionNode message + field 10 on MarketImplicationCard - Seeder: prompt addition + VALID_IMPACT_TYPES validation (>=2 node gate) - Handler: map transmission_chain snake_case -> transmissionChain camelCase - Client: add normalizeCard() to handle both bootstrap (snake_case) and API (camelCase) paths; applied at both data entry points - Renderer: renderChain() with arrow-separated nodes; delegated click handler collapses same node, replaces on different node click - Tests: 4 cases covering snake/camel conversion, absent field default, handler backward-compat |
||
|
Elie Habib
|
d613e5802f |
fix(intelligence): extend country intel brief cache TTL from 2h to 6h (#2430)
Country intel briefs are grounded in seeded world-state data that changes slowly. A 2h TTL causes unnecessary LLM calls on every seeder cycle; 6h better matches the rate of meaningful context change. |
||
|
Elie Habib
|
81d773d2bb |
feat(intelligence): DeepEar QW-1/QW-3/QW-4 — Polymarket context, dual-model routing, robust JSON (#2421)
* feat(intelligence): DeepEar QW-1/QW-3/QW-4 — Polymarket context injection, dual-model routing, robust JSON parsing QW-1: Inject crowd-calibrated Polymarket/Kalshi odds into deduction prompts - Fetch prediction:markets-bootstrap:v1 from Redis and keyword-score markets against user query; top-7 matches appended as structured context block - Hash prediction context into cache key so odds movements trigger fresh LLM calls - Sanitize market titles with sanitizeHeadline() before prompt injection - deductSituation now uses callLlmReasoning (explicit reasoning-tier routing) QW-3: Dual-model LLM routing via env vars - callLlmTool (LLM_TOOL_PROVIDER/MODEL, default groq) for extraction tasks - callLlmReasoning (LLM_REASONING_PROVIDER/MODEL, default openrouter) for synthesis - Private callLlmProfile factory eliminates duplicated wrapper bodies QW-4: Shared _llm-json.mjs utility for robust LLM JSON parsing - cleanJsonText: strips C-style comments and trailing commas before JSON.parse - extractFirstJsonObject/Array: private extractFirstDelimited walker (was 2x23 LOC) - Removes duplicate function definitions from seed-forecasts.mjs - All JSON.parse call sites in tryParseImpactExpansionCandidate and tryParseStructuredCandidate now use cleanJsonText for comment/comma tolerance * fix(intelligence): address PR #2421 review findings - P1: getCachedJson(PREDICTION_BOOTSTRAP_KEY, true) — seed keys must be read raw (unprefixed); without this the prediction-odds block never activates outside production - P2: validate LLM_TOOL_PROVIDER / LLM_REASONING_PROVIDER env value against PROVIDER_SET before use; log a warning and fall back to defaultProvider on typo instead of silently rerouting - Filter: lower word-length threshold from >3 to >1 so short but critical terms like "war", "oil", "EU", "AI", "Fed", "USD" match prediction markets - Suggestion: bucket yesPrice to 5% increments before building the context string to reduce 1%-movement cache churn (20 bands vs 100) * fix(intelligence): guard empty prediction context header when all titles sanitize to empty |
||
|
Elie Habib
|
564c252d48 |
feat(panels): move FrameworkSelector to AI Market Implications panel (#2394)
* feat(panels): move FrameworkSelector from CountryDeepDivePanel to MarketImplicationsPanel
CountryDeepDivePanel was the only panel where the framework selector
wasn't connected to the AI generation path (the country-intel.ts service
reads the framework independently via getActiveFrameworkForPanel).
MarketImplicationsPanel is a better home for it.
Changes:
- Remove FrameworkSelector + hasPremiumAccess from CountryDeepDivePanel
- Add FrameworkSelector (panelId: 'market-implications') to MarketImplicationsPanel
with note "Applies to next AI regeneration"
- Add 'market-implications' to AnalysisPanelId union type
- fetchMarketImplications() now accepts optional framework param and passes
it as ?framework= query string to the API
- data-loader subscribes to 'market-implications' framework changes and
re-triggers loadMarketImplications(), passing the active framework
* fix(market-implications): cache per-framework to prevent N×1 API calls
Previously: any active framework bypassed the client-side TTL cache
entirely, so 100 users with the same framework = 100 separate API calls
per 10-minute window.
Fix:
- Client: replace single-entry cache with Map<frameworkId, {data, cachedAt}>
so each framework variant is cached separately for 10 min
- API: pass ?frameworkId= (stable ID, not the full systemPromptAppend text)
- Server: reads intelligence:market-implications:v1:{frameworkId} when a
known framework ID is present; falls back to the default key if the
framework-specific key doesn't exist yet
- Proto: add framework_id field (sebuf.http.query) to ListMarketImplicationsRequest
With this, 100 users sharing the same framework hit the server once per
10 min per unique frameworkId — the server's framework-keyed Redis entry
is shared across all of them.
* chore(proto): regenerate OpenAPI spec after ListMarketImplicationsRequest frameworkId field
|
||
|
Elie Habib
|
f783bf2d2d |
fix(intelligence): analytical frameworks follow-up — P1 security + P2 correctness fixes (#2386)
* fix(intelligence): include framework/systemAppend hash in cache keys (todos 041, 045, 051) * fix(intelligence): gate framework/systemAppend on server-side PRO check (todo 042) * fix(skills): exact hostname allowlist + redirect:manual to prevent SSRF (todos 043, 054) * fix(intelligence): sanitize systemAppend against prompt injection before LLM (todo 044) * fix(intelligence): use framework field in DeductionPanel, fix InsightsPanel double increment (todos 046, 047) * fix(intelligence): settings export, hot-path cache, country-brief debounce (todos 048, 049, 050) * fix(intelligence): i18n, FrameworkSelector note, stripThinkingTags dedup, UUID IDs (todos 052, 055, 056, 057) - i18n Analysis Frameworks settings section (en + fr locales, replace all hardcoded English strings with t() calls) - FrameworkSelector: replace panelId==='insights' hardcode with note? option; both InsightsPanel and DailyMarketBriefPanel pass note - stripThinkingTags: remove inline duplicate in summarize-article.ts, import from _shared/llm; add Strip unterminated comment so tests can locate the section - Replace Date.now() IDs for imported frameworks with crypto.randomUUID() - Drop 'not supported in phase 1' phrasing to 'not supported' - test: fix summarize-reasoning Fix 2 suite to read from llm.ts - test: add premium-check-stub and wire into redis-caching country intel brief importPatchedTsModule so test can resolve the new import * fix(security): address P1 review findings from PR #2386 - premium-check: require `required: true` from validateApiKey so trusted browser origins (worldmonitor.app, Vercel previews, localhost) are not treated as PRO callers; fixes free-user bypass of framework/systemAppend gate - llm: replace weak sanitizeSystemAppend with sanitizeForPrompt from llm-sanitize.js; all callLlm callers now get model-delimiter and control-char stripping, not just phrase blocklist - get-country-intel-brief: apply sanitizeForPrompt to contextSnapshot before injecting into user prompt; fixes unsanitized query-param injection Closes todos 060, 061, 062 (P1 — blocked merge of #2386). * chore(todos): mark P1 todos 060-062 complete * fix(agentskills): address Greptile P2 review comments - hoist ALLOWED_AGENTSKILLS_HOSTS Set to module scope (was reallocated per-request) - add res.type === 'opaqueredirect' check alongside the 3xx guard; Edge Runtime returns status=0 for opaque redirects so the status range check alone is dead code |
||
|
Elie Habib
|
1f56afeb82 |
feat(panels): disease outbreaks panel/layer, social velocity panel, shipping stress tab (#2383)
* feat(panels): disease outbreaks panel/layer, social velocity panel, shipping stress tab - DiseaseOutbreaksPanel: feed-style panel with alert/warning/watch filter pills, source links, relative timestamps (WHO/ProMED/HealthMap) - SocialVelocityPanel: ranked Reddit trending posts by velocity score with subreddit badge, vote/comment counts, velocity bar - SupplyChainPanel: Stress tab with composite stress gauge and carrier table with sparklines (GetShippingStressResponse) - diseaseOutbreaks map layer: ScatterplotLayer via country centroids, color/radius by alert level, tooltip - MapContainer.setDiseaseOutbreaks(): cached setter with DeckGLMap delegation - data-loader: loadDiseaseOutbreaks/loadSocialVelocity/loadSupplyChain with stress wired into tasks - MapLayers.diseaseOutbreaks added to types, layer registry (globe icon), full variant order, all default objects 🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.com/claude-code) + Compound Engineering v2.49.0 Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com> * fix(supply-chain): add upstreamUnavailable to ShippingStressResponse, restore test-compatible banner guard * fix(panels): filter pills use alertLevel equality, sanitizeUrl on hrefs, globe TODO, E2E layer enabled --------- Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com> |
||
|
Elie Habib
|
110ab402c4 |
feat(intelligence): analytical framework selector for AI panels (#2380)
* feat(frameworks): add settings section and import modal - Add Analysis Frameworks group to preferences-content.ts between Intelligence and Media sections - Per-panel active framework display (read-only, 4 panels) - Skill library list with built-in badge, Rename and Delete actions for imported frameworks - Import modal with two tabs: From agentskills.io (fetch + preview) and Paste JSON - All error cases handled inline: network, domain validation, missing instructions, invalid JSON, duplicate name, instructions too long, rate limit - Add api/skills/fetch-agentskills.ts edge function (proxy to agentskills.io) - Add analysis-framework-store.ts (loadFrameworkLibrary, saveImportedFramework, deleteImportedFramework, renameImportedFramework, getActiveFrameworkForPanel) - Add fw-* CSS classes to main.css matching dark panel aesthetic * feat(panels): wire analytical framework store into InsightsPanel, CountryDeepDive, DailyMarketBrief, DeductionPanel - InsightsPanel: append active framework to geoContext in updateFromClient(); subscribe in constructor, unsubscribe in destroy() - CountryIntelManager: pass framework as query param to fetchCountryIntelBrief(); subscribe to re-open brief on framework change; unsubscribe in destroy() - DataLoaderManager: add dailyBriefGeneration counter for stale-result guard; pass frameworkAppend to buildDailyMarketBrief(); subscribe to framework changes to force refresh; unsubscribe in destroy() - daily-market-brief service: add frameworkAppend? field to BuildDailyMarketBriefOptions; append to extendedContext before summarize call - DeductionPanel: append active framework to geoContext in handleSubmit() before RPC call * feat(frameworks): add FrameworkSelector UI component - Create FrameworkSelector component with premium/locked states - Premium: select dropdown with all framework options, change triggers setActiveFrameworkForPanel - Locked: disabled select + PRO badge, click calls showGatedCta(FREE_TIER) - InsightsPanel: adds asterisk note (client-generated analysis hint) - Wire into InsightsPanel, DailyMarketBriefPanel, DeductionPanel (via this.header) - Wire into CountryDeepDivePanel header right-side (no Panel base, panel=null) - Add framework-selector CSS to main.css * fix(frameworks): make new proto fields optional in generated types * fix(frameworks): extract firstMsg to satisfy strict null checks in tsconfig.api.json * fix(docs): add blank lines around lists/headings to pass markdownlint * fix(frameworks): add required proto string fields to call sites after make generate * chore(review): add code review todos 041-057 for PR #2380 7 review agents (TypeScript, Security, Architecture, Performance, Simplicity, Agent-Native, Learnings) identified 17 findings across 5 P1, 8 P2, and 4 P3 categories. |
||
|
Elie Habib
|
1e1f377078 |
feat(panels): Disease Outbreaks, Shipping Stress, Social Velocity, nuclear test site enrichment (#2375)
* feat(panels): Disease Outbreaks, Shipping Stress, Social Velocity, nuclear test site monitoring - Add HealthService proto with ListDiseaseOutbreaks RPC (WHO + ProMED RSS) - Add GetShippingStress RPC to SupplyChainService (Yahoo Finance carrier ETFs) - Add GetSocialVelocity RPC to IntelligenceService (Reddit r/worldnews + r/geopolitics) - Enrich earthquake seed with Haversine nuclear test-site proximity scoring - Add 5 nuclear test sites to NUCLEAR_FACILITIES (Punggye-ri, Lop Nur, Novaya Zemlya, Nevada NTS, Semipalatinsk) - Add shipping stress + social velocity seed loops to ais-relay.cjs - Add seed-disease-outbreaks.mjs Railway cron script - Wire all new RPCs: edge functions, handlers, gateway cache tiers, health.js STANDALONE_KEYS/SEED_META * fix(relay): apply gold standard retry/TTL-extend pattern to shipping-stress and social-velocity seeders * fix(review): address all PR #2375 review findings - health.js: shippingStress maxStaleMin 30→45 (3x interval), socialVelocity 20→30 (3x interval) - health.js: remove shippingStress/diseaseOutbreaks/socialVelocity from ON_DEMAND_KEYS (relay/cron seeds, not on-demand) - cache-keys.ts: add shippingStress, diseaseOutbreaks, socialVelocity to BOOTSTRAP_CACHE_KEYS - ais-relay.cjs: stressScore formula 50→40 (neutral market = moderate, not elevated) - ais-relay.cjs: fetchedAt Date.now() (consistent with other seeders) - ais-relay.cjs: deduplicate cross-subreddit article URLs in social velocity loop - seed-disease-outbreaks.mjs: WHO URL → specific DON RSS endpoint (not dead general news feed) - seed-disease-outbreaks.mjs: validate() requires outbreaks.length >= 1 (reject empty array) - seed-disease-outbreaks.mjs: stable id using hash(link) not array index - seed-disease-outbreaks.mjs: RSS regexes use [\s\S]*? for CDATA multiline content - seed-earthquakes.mjs: Lop Nur coordinates corrected (41.39,89.03 not 41.75,88.35) - seed-earthquakes.mjs: sourceVersion bumped to usgs-4.5-day-nuclear-v1 - earthquake.proto: fields 8-11 marked optional (distinguish not-enriched from enriched=false/0) - buf generate: regenerate seismology service stubs * revert(cache-keys): don't add new keys to bootstrap without frontend consumers * fix(panels): address all P1/P2/P3 review findings for PR #2375 - proto: add INT64_ENCODING_NUMBER annotation + sebuf import to get_shipping_stress.proto (run make generate) - bootstrap: register shippingStress (fast), socialVelocity (fast), diseaseOutbreaks (slow) in api/bootstrap.js + cache-keys.ts - relay: update WIDGET_SYSTEM_PROMPT with new bootstrap keys and live RPCs for health/supply-chain/intelligence - seeder: remove broken ProMED feed URL (promedmail.org/feed/ returns HTML 404); add 500K size guard to fetchRssItems; replace private COUNTRY_CODE_MAP with shared geo-extract.mjs; remove permanently-empty location field; bump sourceVersion to who-don-rss-v2 - handlers: remove dead .catch from all 3 new RPC handlers; fix stressLevel fallback to low; fix fetchedAt fallback to 0 - services: add fetchShippingStress, disease-outbreaks.ts, social-velocity.ts with getHydratedData consumers |
||
|
Elie Habib
|
af49e5563b |
fix(feeds): replace dead direct RSS URLs with Google News proxies (#2360)
- dlnews.com/feed/ → 404 (src/config/feeds.ts + server/_feeds.ts) - news.usni.org/feed → 403 (server/_feeds.ts) Relay logs show these hitting every poll cycle with backoff climbing. Google News site-search gives equivalent coverage without the upstream block. |
||
|
Elie Habib
|
015ce0aba5 |
fix(llm): strip markdown fences + skip groq in analyze-stock (#2359)
- Strip ```json ... ``` fences from all LLM responses in callLlm before validation — fixes Gemini 2.5 Flash via OpenRouter failing validate() and silently falling back to rules-based overlay on every request - Add providerOrder: ['openrouter', 'generic'] to analyze-stock callLlm to skip Groq entirely (consistently 429) and go direct to OpenRouter |
||
|
Elie Habib
|
e3b863d30f |
feat(panels): EU data tabs for Energy, Macro, Yield, Commodities panels (#2355)
* feat(panels): add EU data tabs to EnergyComplex, MacroTiles, YieldCurve, Commodities - EnergyComplexPanel: US Nat Gas Storage (EIA) + EU Gas Storage (GIE AGSI+) sections below crude - MacroTilesPanel: US/EU tab toggle; EU shows avg HICP/unemployment/GDP for DE/FR/IT/ES + ECB ESTR - YieldCurvePanel: Curve/ECB Rates tab; Rates shows ESTR + EURIBOR 3M/6M/1Y sparklines via FRED - CommoditiesPanel: Commodities/EUR FX tab; FX shows ECB EUR pairs (USD/GBP/JPY/CHF/CAD/CNY/AUD) - Add getEuGasStorageData() + getEurostatCountryData() service functions with circuit breakers - Register eurostatCountryData in bootstrap.js + cache-keys.ts (SLOW tier) - Wire fetchNatGasStorageRpc, getEuGasStorageData, getEcbFxRatesData in data-loader.loadOilAnalytics * fix(commodities-panel): always re-render on data update so FX tab bar appears |
||
|
Elie Habib
|
5d68f0ae6b |
fix(intelligence): land news:threat:summary:v1 CII work missed from PR #2096 (#2356)
* feat(intelligence): emit news:threat:summary:v1 from relay classify loop for CII
During seedClassifyForVariant(), attribute each title to ISO2 countries
while both title and classification result are in scope. At the end of
seedClassify(), merge per-country threat counts across all variants and
write news:threat:summary:v1 (20min TTL) with { byCountry: { [iso2]: {
critical, high, medium, low, info } }, generatedAt }.
get-risk-scores.ts reads the new key via fetchAuxiliarySources() and
applies weighted scores (critical→4, high→2, medium→1, low→0.5, info→0,
capped at 20) per country into the information component of CII eventScore.
Closes #2053
* fix(intelligence): register news:threat-summary in health.js and expand tests
- Add newsThreatSummary to BOOTSTRAP_KEYS (seed-meta:news:threat-summary,
maxStaleMin: 60) so relay classify outages surface in health dashboard
- Add 4 tests: boost verification, cap-at-20, unknown-country safety,
null-threatSummary zero baseline
* fix(classify): de-dup cross-variant titles and attribute to last-mentioned country
P1-A: seedClassify() was summing byCountry across all 5 variants (full/tech/
finance/happy/commodity) without de-duplicating. Shared feeds (CNBC, Yahoo
Finance, FT, HN, Ars) let a single headline count up to 4x before reaching
CII, saturating threatSummaryScore on one story.
Fix: pass seenTitles Set into seedClassifyForVariant; skip attribution for
titles already counted by an earlier variant.
P1-B: matchCountryNamesInText() was attributing every country mentioned in a
headline equally. "UK and US launch strikes on Yemen" raised GB, US, and YE
with identical weight, inflating actor-country CII.
Fix: return only the last country in document order — the grammatical object
of the headline, which is the primary affected country in SVO structure.
* fix(classify): replace last-position heuristic with preposition-pattern attribution
The previous "last-mentioned country" fix still failed for:
- "Yemen says UK and US strikes hit Hodeidah" → returned US (wrong)
- "US strikes on Yemen condemned by Iran" → returned IR (wrong)
Both failures stem from position not conveying grammatical role. Switch to a
preposition/verb-pattern approach: only attribute to a country that immediately
follows a locative preposition (in/on/against/at/into/targeting/toward) or an
attack verb (invades/attacks/bombs/hits/strikes). No pattern match → return []
(skip attribution rather than attribute to the wrong country).
* fix(classify): fix regex hitting, gaza/hamas geo mapping, seed-meta always written
- hitt?(?:ing|s)? instead of hit(?:s|ting)? so "hitting" is matched
- gaza → PS (Palestinian Territories), hamas → PS (was IL)
- seed-meta:news:threat-summary written unconditionally so health check
does not fire false alerts during no-attribution runs
|
||
|
Elie Habib
|
9480b547d5 |
feat(feeds): US Natural Gas Storage weekly seeder (EIA NW2_EPG0_SWO_R48_BCF) (#2353)
* feat(feeds): US Natural Gas Storage seeder via EIA (NW2_EPG0_SWO_R48_BCF)
Adds weekly EIA natural gas working gas storage for the Lower-48 states
(series NW2_EPG0_SWO_R48_BCF, in Bcf), mirroring the crude inventories
pattern exactly. Companion dataset to EU gas storage (GIE AGSI+).
- proto: GetNatGasStorage RPC + NatGasStorageWeek message
- seed-economy.mjs: fetchNatGasStorage() in Promise.allSettled, writes
economic:nat-gas-storage:v1 with 21-day TTL (3x weekly cadence)
- server handler: getNatGasStorage reads seeded key from Redis
- gateway: /api/economic/v1/get-nat-gas-storage → static tier
- health.js: BOOTSTRAP_KEYS + SEED_META (14-day maxStaleMin)
- bootstrap.js: KEYS + SLOW_KEYS
- cache-keys.ts: BOOTSTRAP_CACHE_KEYS + BOOTSTRAP_TIERS (slow)
* feat(feeds): add fetchNatGasStorageRpc consumer in economic service
Adds getHydratedData('natGasStorage') consumer required by bootstrap
key registry test, plus circuit breaker and RPC wrapper mirroring
fetchCrudeInventoriesRpc pattern.
|
||
|
Elie Habib
|
4438ef587f |
feat(feeds): ECB CISS European financial stress index (#2278) (#2334)
* feat(feeds): ECB CISS European financial stress seeder + GetEuFsi RPC (#2278) - New seed-fsi-eu.mjs fetches ECB CISS (0-1 systemic stress index for Euro area) via SDMX-JSON REST API (free, no auth); TTL=604800s (7d, weekly data cadence) - New GetEuFsi RPC in EconomicService with proto + handler; cache tier: slow - FSIPanel now shows EU CISS gauge below US FSI with label thresholds: Low<0.2, Moderate<0.4, Elevated<0.6, High>=0.6 - Registered economic:fsi-eu:v1 in health.js BOOTSTRAP_KEYS + SEED_META, bootstrap.js, cache-keys.ts BOOTSTRAP_TIERS; hydrated via getHydratedData('euFsi') - All 2348 test:data tests pass; typecheck + typecheck:api clean * fix(ecb-ciss): address code review findings on PR #2334 - Raise FSI_EU_TTL from 604800s (7d) to 864000s (10d) to match other weekly seeds (bigmac, groceryBasket, fuelPrices) and provide a 3-day buffer against cron-drift or missed Saturday runs - Format latestDate via toLocaleDateString() in FSIPanel CISS section instead of displaying the raw ISO string (e.g. "2025-04-04") * fix(ecb-ciss): address Greptile review comments on PR #2334 - Fix misleading "Daily frequency" comment in seed-fsi-eu.mjs (SDMX uses 'D' series key but only Friday/weekly observations are present) - Replace latestValue > 0 guards with Number.isFinite() in FSIPanel.ts so a valid CISS reading of exactly 0 is not incorrectly excluded * chore: regenerate proto outputs after rebase |
||
|
Elie Habib
|
1d0846fa98 |
feat(feeds): ECB Euro Area yield curve seeder (#2276) (#2330)
* feat(feeds): ECB Euro Area yield curve seeder — EU complement to US Treasury curve (#2276) - Add scripts/seed-yield-curve-eu.mjs: fetches ECB AAA sovereign spot rates (1Y-30Y) via ECB Data Portal SDMX-JSON, writes to economic:yield-curve-eu:v1, TTL=259200s (3x daily interval) - Add proto/worldmonitor/economic/v1/get_eu_yield_curve.proto + GetEuYieldCurve RPC - Add server/worldmonitor/economic/v1/get-eu-yield-curve.ts handler (reads from Redis cache) - Wire GetEuYieldCurve into handler.ts and gateway.ts (daily cache tier) - Update YieldCurvePanel.ts: fetches EU curve in parallel with US FRED data, overlays green dashed line on chart with ECB AAA legend entry - Update api/health.js: add euYieldCurve to BOOTSTRAP_KEYS + SEED_META (maxStaleMin=2880, daily seed) - Re-run buf generate to update generated client/server TypeScript types * fix(generated): restore @ts-nocheck in economic generated files after buf regeneration * fix(ecb-yield-curve): address Greptile review comments on PR #2330 Include priorValues in allValues for yMin/yMax scale computation so prior US curve polyline is never clipped outside the chart area. * fix(generated): regenerate OpenAPI docs after rebase conflict resolution Re-run make generate to fix schema ordering in EconomicService.openapi.yaml and .json after manual conflict resolution introduced ordering inconsistencies. * chore: regenerate proto outputs after rebase |
||
|
Elie Habib
|
c8397b1f7e |
feat(feeds): ECB ESTR + EURIBOR short-rate seeder (#2279) (#2335)
* feat(feeds): ECB ESTR + EURIBOR seeder — EU short-rate complement to SOFR/Fed Funds (#2279) * fix(feeds): address Greptile P2 review comments on ECB short-rate seeder - Bump lock TTL from 120s to 300s to cover worst-case full-retry run (~244s) - Add multi-series warning in parseSdmxJson when ECB returns >1 series key |
||
|
Elie Habib
|
b6847e5214 |
feat(feeds): GIE AGSI+ EU gas storage seeder (#2281) (#2339)
* feat(feeds): GIE AGSI+ EU gas storage seeder — European energy security indicator (#2281) - New scripts/seed-gie-gas-storage.mjs: fetches EU aggregate gas storage fill % from GIE AGSI+ API, computes 1-day change, trend (injecting/withdrawing/stable), and days-of-consumption estimate; TTL=259200s (3x daily); isMain guard; CHROME_UA; validates fillPct in (0,100]; graceful degradation when GIE_API_KEY absent - New proto/worldmonitor/economic/v1/get_eu_gas_storage.proto + GetEuGasStorage RPC wired into EconomicService - New server/worldmonitor/economic/v1/get-eu-gas-storage.ts handler (reads seeded Redis key) - api/health.js: BOOTSTRAP_KEYS + SEED_META (maxStaleMin=2880, 2x daily cadence) - api/bootstrap.js: euGasStorage key in SLOW_KEYS bucket - Regenerated src/generated/ + docs/api/ via make generate * fix(feeds): wire euGasStorage into cache-keys, gateway tier, and test PENDING_CONSUMERS - server/_shared/cache-keys.ts: add euGasStorage → economic:eu-gas-storage:v1 (slow tier) - server/gateway.ts: add /api/economic/v1/get-eu-gas-storage → slow RPC_CACHE_TIER - tests/bootstrap.test.mjs: add euGasStorage to PENDING_CONSUMERS (no frontend panel yet) * fix(gie-gas-storage): normalize seededAt to string to match proto int64 contract Proto int64 seeded_at maps to string in JS; seed was writing Date.now() (number). Fix seed to write String(Date.now()) and add handler-side normalization for any stale Redis entries that may have the old numeric format. * fix(feeds): coerce nullable fillPctChange1d/gasDaysConsumption to 0 (#2281) Greptile P1: both fields could be null (single data-point run or missing volume) but the proto interface declares them as non-optional numbers. Seed script now returns 0 instead of null; handler defensively coerces nulls from older cached blobs via nullish coalescing. Dead null-guard on trend derivation also removed. |
||
|
Elie Habib
|
c69a13a1a0 |
feat(feeds): Eurostat per-country economic data seeder (#2282) (#2340)
* feat(feeds): Eurostat per-country CPI/unemployment/GDP seeder for 10 EU member states (#2282) * fix(feeds): address Greptile P1/P2 issues in eurostat seed parser - P1: fix sparse-index iteration bug in parseEurostatResponse; loop now iterates over Object.keys(values) directly instead of using key count as sequential bound, correctly handling non-zero-based sparse indexes - P2: remove unused dimIdx, geoStride, timeStride, totalSize variables - P2: batch country fetches (3 at a time) to reduce peak Eurostat concurrency from 30 to 9 simultaneous requests * chore: regenerate EconomicService openapi JSON after rebase |
||
|
Elie Habib
|
b5faffb341 |
feat(feeds): ECB reference FX rates seeder (#2280) (#2337)
* feat(feeds): ECB daily reference FX rates seeder -- EUR/USD/GBP/JPY/CHF (#2280) - New scripts/seed-ecb-fx-rates.mjs: fetches EUR/USD/GBP/JPY/CHF/CAD/AUD/CNY daily from ECB Data Portal (no API key required) - New getEcbFxRates RPC in economic service (proto + handler + gateway slow tier) - api/health.js: ecbFxRates in BOOTSTRAP_KEYS + SEED_META (2880min maxStale) - api/bootstrap.js: ecbFxRates registered as SLOW_KEYS bootstrap entry - TTL: 259200s (3x daily interval), isMain guard, CHROME_UA, validate>=3 pairs * fix(feeds): register ecbFxRates in cache-keys.ts + add frontend hydration consumer * fix(ecb-fx-rates): use dynamic CURRENCY dim position in series key parsing CURRENCY is at index 1 in EXR series keys (FREQ:CURRENCY:CURRENCY_DENOM:EXR_TYPE:EXR_SUFFIX). Using hardcoded keyParts[0] always read the FREQ field (always "0"), so all series resolved to currencyCodes[0] (AUD) and only 1 pair was written to Redis instead of all 7. Fix: find CURRENCY position via findIndex() and use keyParts[currencyDimPos] to extract the correct per-series currency index. * fix(feeds): hoist obs-dimension lookup + fix ECB UTC publication time - Hoist obsPeriods/timeDim/timeValues above the series loop (loop-invariant, P2) - Fix updatedAt timestamp: ECB publishes at 16:00 CET = 14:00 UTC (not 16:00 UTC), use T14:00:00Z (P2) * chore(generated): fix EcbFxRate schema ordering in EconomicService openapi.json buf generate places EcbFxRate alphabetically before EconomicEvent; align committed file with code-generator output so proto freshness hook passes. |
||
|
Elie Habib
|
f3b0280227 |
feat(economic): EIA weekly crude oil inventory seeder (#2142) (#2168)
* feat(economic): EIA weekly crude oil inventory seeder (#2142) - scripts/seed-economy.mjs: add fetchCrudeInventories() fetching WCRSTUS1, compute weeklyChangeMb, write economic:crude-inventories:v1 (10-day TTL) - proto/worldmonitor/economic/v1/get_crude_inventories.proto: new proto with CrudeInventoryWeek and GetCrudeInventories RPC - server/worldmonitor/economic/v1/get-crude-inventories.ts: RPC handler reading seeded key with getCachedJson(..., true) - server/worldmonitor/economic/v1/handler.ts: wire in getCrudeInventories - server/gateway.ts: add static cache tier for /api/economic/v1/get-crude-inventories - api/health.js: crudeInventories in BOOTSTRAP_KEYS + SEED_META (maxStaleMin: 20160, 2x weekly cadence) - src/services/economic/index.ts: add fetchCrudeInventoriesRpc() with circuit breaker - src/components/EnergyComplexPanel.ts: surface 8-week sparkline and WoW change in energy panel - src/app/data-loader.ts: call fetchCrudeInventoriesRpc() in loadOilAnalytics() * fix: remove stray market-implications gateway entry from crude-inventories branch * fix(crude-inventories): address ce-review P1/P2 findings before merge - api/bootstrap.js: register crudeInventories in BOOTSTRAP_CACHE_KEYS + SLOW_KEYS (P1-001) - server/_shared/cache-keys.ts: add crudeInventories key + tier to match bootstrap.js - api/health.js: remove bundled marketImplications (belongs in separate PR) (P1-002) - src/services/economic/index.ts: add isFeatureAvailable('energyEia') gate (P2-003) - src/services/economic/index.ts: use getHydratedData('crudeInventories') on first load - proto/get_crude_inventories.proto: weekly_change_mb → optional double (P2-004) - scripts/seed-economy.mjs: CRUDE_INVENTORIES_TTL 10d → 21d (3× cadence) (P2-005) - scripts/seed-economy.mjs: period format validation with YYYY-MM-DD regex (P3-007) - src/app/data-loader.ts: warn on crude fetch rejection (P2-006) * fix(crude-inventories): schema validation, MIN_ITEMS gate, handler logging, raw=true docs - Handler: document raw=true param, log errors instead of silent catch - Seeder: CRUDE_MIN_WEEKS=4 guard prevents quota-hit empty writes - Seeder: isValidWeek() schema validation before Redis write * chore: regenerate openapi docs after rebase (adds getCrudeInventories + getEconomicCalendar) * fix(gateway): add list-market-implications to RPC_CACHE_TIER * chore: exclude todos/ from markdownlint |
||
|
Elie Habib
|
d80736d1ba |
feat(heatmap): add sorted bar chart view to sector HeatmapPanel (#2326)
* feat(heatmap): add sorted bar chart view to HeatmapPanel (#2246) - Add FearGreedSectorPerformance message to proto + regenerate - Expose sectorPerformance array in GetFearGreedIndexResponse RPC handler - HeatmapPanel.renderHeatmap accepts optional sectorBars; renders sorted horizontal bar chart below existing tile grid - Sectors ranked by day change (gainers first, losers last) - Green/red bars proportional to |change| (max ~3% = 100% width) - Reuses fearGreedIndex bootstrap data — zero extra network calls - Add .heatmap-bar-chart CSS: 18px rows, 10px font, compact layout * fix(heatmap): address code review P1 findings on PR #2326 - Server: guard change1d with Number.isFinite fallback so non-numeric Redis values (NaN) coerce to 0 instead of propagating through - Client: filter out any NaN change1d entries before computing maxAbs; Math.max propagates NaN so a single bad entry makes all bars invisible (width:NaN%); also bail early if filtered list is empty * fix(heatmap): address Greptile review comments on PR #2326 Use var(--green)/var(--red) for bar fill colour to match text label CSS variables and avoid visual mismatch across themes. |
||
|
Elie Habib
|
dc5061a8ac |
feat(intelligence): Tier 1 brief improvements - infrastructure context, structured format, conflict counter, risk scores (#2338)
* feat(intelligence): Tier 1 brief improvements from improvement plan
- Restructure country intel brief prompt to output structured sections
(SITUATION NOW, WHAT THIS MEANS FOR [COUNTRY], KEY RISKS, OUTLOOK,
WATCH ITEMS) replacing generic 5-paragraph prose narrative
- Add buildInfrastructureContext() to country-intel.ts: assembles named
cables, pipelines, ports and chokepoints from the dependency graph into
the contextSnapshot so the LLM can produce entity-specific impact bullets
- Add conflict-day-counter.ts: static config of active conflicts with
running day counts, rendered as a mono bar in the header (full variant)
- Add numeric risk score badge (0-100) to news event cards using
threat-level + confidence derivation; setRiskScoreGetter() allows the
app to wire in the full CII + geo-convergence formula later
- Update formatBrief() in all three brief renderers (CountryIntelModal,
CountryBriefPage, CountryDeepDivePanel) to parse and style the new
structured sections with CSS classes brief-section, brief-bullet,
brief-outlook-row
* fix(intelligence): review fixes for brief improvement plan
- Bump cache key v2→v3 to invalidate old prose-format responses
- Inject \${countryName} into prompt instead of literal [COUNTRY] placeholder
- Move infrastructure context to TOP of contextSnapshot so it survives
the 2200→3800 char limit increase (was silently dropped for active countries)
- Extract formatBrief() to shared src/utils/format-intel-brief.ts,
eliminating triplication across CountryIntelModal/CountryBriefPage/CountryDeepDivePanel
- Wire setRiskScoreGetter() in both NewsPanel creation paths in panel-layout.ts
using 0.57×severity + 0.43×geoConvergence (CII pending lat/lon→country lookup)
* test: update cache key assertion v2→v3 after prompt rewrite
* fix(intelligence): runtime type guard for pipeline status, add context limit comment
|
||
|
Elie Habib
|
caf2e8fa1a |
fix(cors): add X-Widget-Key + X-Pro-Key to _cors.js; reduce Max-Age 86400→3600 (#2321)
86400s max-age cached old preflight results in browsers for 24h — any CORS header change left users broken until the cache expired. 3600 (1h) is safer. Also sync api/_cors.js with server/cors.ts — it was missing X-Widget-Key and X-Pro-Key, which are needed for widget-agent cross-origin requests. |
||
|
Elie Habib
|
fc58659161 | fix(cors): add X-Widget-Key to Access-Control-Allow-Headers (#2314) | ||
|
Elie Habib
|
76ba102220 | debug(auth): expose full key comparison detail in Invalid API key 401 (#2313) | ||
|
Elie Habib
|
99b414bc5e |
fix(auth): remove 'convex' JWT template dependency for premium RPCs (#2308)
* fix(auth): remove 'convex' JWT template dependency for premium RPCs Two changes to make premium panel auth work without requiring a custom Clerk JWT template: Client (clerk.ts): - getClerkToken() now tries the 'convex' template first, then falls back to the standard session token if the template isn't configured in Clerk dashboard or returns null Server (auth-session.ts): - Remove audience:'convex' from jwtVerify — accepts both template tokens (aud='convex') and standard Clerk session tokens; the issuer check alone prevents cross-app token reuse - When the JWT lacks a `plan` claim (standard token path), look up the user's plan via the Clerk Backend API using CLERK_SECRET_KEY - Cache plan lookups in-memory (5min TTL) to avoid per-request Clerk API calls Fixes premium Stock Analysis and Backtesting panels returning 401 for Pro users on deployments where the 'convex' JWT template was absent. * test(auth-session): update audience test to reflect removed audience check |
||
|
Elie Habib
|
d4917cf00b |
feat(military): USPTO PatentsView defense/dual-use patent filing seeder (#2047) (#2091)
* feat(military): USPTO PatentsView defense/dual-use patent seeder (#2047) * fix(military): correct PatentsView API field names, sort param, and total semantics - Use nested dot-notation field names required by PatentsView v1 API: assignees.assignee_organization (was flat assignee_organization) cpc_at_issue.cpc_subclass_id (was non-existent cpc_subgroup_id) - Split sort into separate s= param; fix per_page -> size in o= param (sort inside o= and per_page are both rejected by PatentsView v1) - Fix listDefensePatents total to reflect full seeded dataset size, not the post-filter count (proto doc: "Total number of filings in the seeded dataset") * feat(defense-patents): add DefensePatentsPanel — R&D Signal frontend Panel displays weekly USPTO defense/dual-use patent filings from seed. Five tabs: All | Comms (H04B) | Semiconductors (H01L) | Ammunition (F42B) | AI (G06N) | Biotech (C12N). Shows assignee, title, CPC tag, date, USPTO link. Wired into panel-layout.ts and registered as 'defense-patents' in panels.ts. * fix(defense-patents): TTL 3x interval, recordCount, maxStaleMin 2x, wire into App.ts - CACHE_TTL: 7d → 21d (3× weekly interval) so key survives missed runs - recordCount: add (d) => d?.patents?.length ?? 0 so seed logs correct count - health.js maxStaleMin: 10080 → 20160 (2× 7-day interval per gold standard) - App.ts: import DefensePatentsPanel, add primeTask (initial load) and scheduleRefresh at 24h interval (daily poll for weekly data) |
||
|
Elie Habib
|
a969a9e3a3 |
feat(auth): integrate clerk.dev (#1812)
* feat(auth): integrate better-auth with @better-auth/infra dash plugin Wire up better-auth server config with the dash() plugin from @better-auth/infra, and the matching sentinelClient() on the client side. Adds BETTER_AUTH_API_KEY to .env.example. * feat(auth): swap @better-auth/infra for @convex-dev/better-auth [10-01 task 1] Install @convex-dev/better-auth@0.11.2, remove @better-auth/infra, delete old server/auth.ts skeleton, rewrite auth-client.ts to use crossDomainClient + convexClient plugins. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): create Convex auth component files [10-01 task 2] Add convex.config.ts (register betterAuth component), auth.config.ts (JWT/JWKS provider), auth.ts (better-auth server with Convex adapter, crossDomain + convex plugins), http.ts (mount auth routes with CORS). Uses better-auth/minimal for lighter bundle. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add admin, organization, and dash plugins [10-01] Re-install @better-auth/infra for dash() plugin to enable dash.better-auth.com admin dashboard. Add admin() and organization() plugins from better-auth/plugins for user and org management. Update both server (convex/auth.ts) and client (auth-client.ts). Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): drop @better-auth/infra (Node.js deps incompatible with Convex V8) Keep admin() and organization() from better-auth/plugins (V8-safe). @better-auth/infra's dash() transitively imports SAML/SSO with node:crypto, fs, zlib — can't run in Convex's serverless runtime. Dashboard features available via admin plugin endpoints instead. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(11-01): create auth-state.ts with OTT handler and session subscription - Add initAuthState() for OAuth one-time token verification on page load - Add subscribeAuthState() reactive wrapper around useSession nanostore atom - Add getAuthState() synchronous snapshot getter - Export AuthUser and AuthSession types for UI consumption Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(11-01): add Google OAuth provider and wire initAuthState into App.ts - Add socialProviders.google with GOOGLE_CLIENT_ID/SECRET to convex/auth.ts - Add all variant subdomains to trustedOrigins for cross-subdomain CORS - Call initAuthState() in App.init() before panelLayout.init() - Add authModal field to AppContext interface (prepares for Plan 02) - Add authModal: null to App constructor state initialization Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(11-02): create AuthModal with Sign In/Sign Up tabs and Google OAuth - Sign In tab: email/password form calling authClient.signIn.email() - Sign Up tab: name/email/password form calling authClient.signUp.email() - Google OAuth button calling authClient.signIn.social({ provider: 'google', callbackURL: '/' }) - Auto-close on successful auth via subscribeAuthState() subscription - Escape key, overlay click, and X button close the modal - Loading states, error display, and client-side validation Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(11-02): add AuthHeaderWidget, mount in header, add auth CSS - AuthHeaderWidget: reactive header widget showing Sign In button (anonymous) or avatar + dropdown (authenticated) - User dropdown: name, email, Free tier badge, Sign Out button calling authClient.signOut() - setupAuthWidget() in EventHandlerManager creates modal + widget, mounts at authWidgetMount span - authWidgetMount added to panel-layout.ts header-right, positioned before download wrapper - setupAuthWidget() called from App.ts after setupUnifiedSettings() - Full auth CSS: modal styles, tabs, forms, Google button, header widget, avatar, dropdown Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(11-02): add localhost:3000 to trustedOrigins for local dev CORS Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): remove admin/organization plugins that break Convex adapter validator The admin() plugin adds banned/role fields to user creation data, but the @convex-dev/better-auth adapter validator doesn't include them. These plugins are Phase 12 work — will re-add with additionalFields config when needed. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(12-01): add Resend email transport, verification + reset callbacks, role field - Install resend SDK for transactional email - Add emailVerification with sendOnSignUp:true and fire-and-forget Resend callbacks - Add sendResetPassword callback with 1-hour token expiry - Add user.additionalFields.role (free/pro, input:false, defaultValue:free) - Create userRoles fallback table in schema with by_userId index - Create getUserRole query and setUserRole mutation in convex/userRoles.ts - Lazy-init Resend client to avoid Convex module analysis error Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(12-01): enhance auth-state with emailVerified and role fields - Add emailVerified (boolean) and role ('free' | 'pro') to AuthUser interface - Fetch role from Convex userRoles table via HTTP query after session hydration - Cache role per userId to avoid redundant fetches - Re-notify subscribers asynchronously when role is fetched for a new user - Map emailVerified from core better-auth user field (default false) - Derive Convex cloud URL from VITE_CONVEX_SITE_URL env var Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(12-01): add Convex generated files from deployment - Track convex/_generated/ files produced by npx convex dev --once Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(12-03): create panel-gating service with auth-aware showGatedCta - Add PanelGateReason enum (NONE/ANONYMOUS/UNVERIFIED/FREE_TIER) - Add getPanelGateReason() computing gating from AuthSession + premium flag - Add Panel.showGatedCta() rendering auth-aware CTA overlays - Add Panel.unlockPanel() to reverse locked state - Extract lockSvg to module-level const shared by showLocked/showGatedCta - Add i18n keys: signInToUnlock, signIn, verifyEmailToUnlock, resendVerification, upgradeDesc, upgradeToPro Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(12-02): add forgot password flow, password reset form, and token detection - Widen authModal interface in app-context.ts to support reset-password mode and setResetToken - AuthModal refactored with 4 views: signin, signup, forgot-password, reset-password - Forgot password view sends reset email via authClient.requestPasswordReset - Reset password form validates matching passwords and calls authClient.resetPassword - auth-state.ts detects ?token= param from email links, stores as pendingResetToken - App.ts routes pending reset token to auth modal after UI initialization - CSS for forgot-link, back-link, and success message elements Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(12-02): add email verification banner to AuthHeaderWidget and tier badge - Show non-blocking verification banner below header for unverified users - Banner has "Resend" button calling authClient.sendVerificationEmail - Banner is dismissible (stored in sessionStorage, reappears next session) - Tier badge dynamically shows Free/Pro based on user.role - Pro badge has gradient styling distinct from Free badge - Dropdown shows unverified status indicator with yellow dot - Banner uses fixed positioning, does not push content down - CSS for banner, pro badge, and verification status indicators Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(12-03): wire reactive auth-based gating into panel-layout - Add WEB_PREMIUM_PANELS Set (stock-analysis, stock-backtest, daily-market-brief) - Subscribe to auth state changes in PanelLayoutManager.init() - Add updatePanelGating() iterating panels with getPanelGateReason() - Add getGateAction() returning CTA callbacks per gate reason - Remove inline showLocked() calls for web premium panels - Preserve desktop _lockPanels for forecast, oref-sirens, telegram-intel - Clean up auth subscription in destroy() Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(13-01): create auth-token utility and inject Bearer header in web fetch redirect - Add src/services/auth-token.ts with getSessionBearerToken() that reads session token from localStorage - Add WEB_PREMIUM_API_PATHS Set for the 4 premium market API paths - Inject Authorization: Bearer header in installWebApiRedirect() for premium paths when session exists - Desktop installRuntimeFetchPatch() left unchanged (API key only) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(13-01): create server-side session validation module - Add server/auth-session.ts with validateBearerToken() for Vercel edge gateway - Validates tokens via Convex /api/auth/get-session with Better-Auth-Cookie header - Falls back to userRoles:getUserRole Convex query for role resolution - In-memory cache with 60s TTL and 100-entry cap - Network errors not cached to allow retry on next request Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(13-02): add bearer token fallback auth for premium API endpoints - Dynamic import of auth-session.ts when premium endpoint + API key fails - Valid pro session tokens fall through to route handler - Non-pro authenticated users get 403 'Pro subscription required' - Invalid/expired tokens get 401 'Invalid or expired session' - Non-premium endpoints and static API key flow unchanged Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): sign-in button invisible in dark theme — white on white --accent is #fff in dark theme, so background: var(--accent) + color: #fff was invisible. Changed to transparent background with var(--text) color. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): add premium panel keys to full and finance variant configs stock-analysis, stock-backtest, and daily-market-brief were defined in the shared panels.ts but missing from variant DEFAULT_PANELS, causing shouldCreatePanel() to return false and panel gating CTAs to never render. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test(auth): add Playwright smoke tests for auth UI (phases 12-13) 6 tests covering: Sign In button visibility, auth modal opening, modal views (Sign In/Sign Up/Forgot Password), premium panel gating for anonymous users, and auth token absence when logged out. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): remove role additionalField that breaks Convex component validator The betterAuth Convex component has a strict input validator for the user model that doesn't include custom fields. The role additionalField caused ArgumentValidationError on sign-up. Roles are already stored in the separate userRoles table — no data loss. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): use Authorization Bearer header for Convex session validation Better-Auth-Cookie header returned null — the crossDomain plugin's get-session endpoint expects Authorization: Bearer format instead. Confirmed via curl against live Convex deployment. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): use verified worldmonitor.app domain for auth emails Was using noreply@resend.dev (testing domain) which can't send to external recipients. Switched to noreply@worldmonitor.app matching existing waitlist/contact emails. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): await Resend email sends — Convex kills dangling promises void (fire-and-forget) causes Convex to terminate the fetch before Resend receives it. Await ensures emails actually get sent. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: update Convex generated auth files after config changes Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): guard against undefined VITE_CONVEX_SITE_URL in auth-state The Convex cloud URL derivation crashed the entire app when VITE_CONVEX_SITE_URL wasn't set in the build environment (Vercel preview). Now gracefully defaults to empty string and skips role fetching when the URL is unavailable. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(auth): add dash + organization plugins, remove Google OAuth, fix dark mode button - Add @better-auth/infra dash plugin for hosted admin dashboard - Add organization plugin for org management in dashboard - Add dash.better-auth.com to trustedOrigins - Remove Google OAuth (socialProviders, button, divider, CSS) - Fix auth submit button invisible in dark mode (var(--accent) → #3b82f6) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): replace dash plugin with admin — @better-auth/infra incompatible with Convex V8 @better-auth/infra imports SSO/SAML libraries requiring Node.js built-ins (crypto, fs, stream) which Convex's V8 runtime doesn't support. Replaced with admin plugin from better-auth/plugins which provides user management endpoints (set-role, list-users, ban, etc.) natively. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: remove stale Convex generated files after plugin update Convex dev regenerated _generated/ — the per-module JS files (auth.js, http.js, schema.js, etc.) are no longer emitted. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore(auth): remove organization plugin — will add in subsequent PR Organization support (team accounts, invitations, member management) is not wired into any frontend flow yet. Removing to keep the auth PR focused on email/password + admin endpoints. Will add back when building the org/team feature. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add authentication & panel gating guide Documents the auth stack, panel gating configuration, server-side session enforcement, environment variables, and user roles. Includes step-by-step guide for adding new premium panels. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(test): stub panel-gating in RuntimeConfigPanel test harness Panel.ts now imports @/services/panel-gating, which wasn't stubbed — causing the real runtime.ts (with window.location) to be bundled, breaking Node.js tests with "ReferenceError: location is not defined". Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): allow Vercel preview origins in Convex trustedOrigins * fix(auth): broaden Convex trustedOrigins to cover *.worldmonitor.app previews * fix(auth): use hostonly wildcard pattern for *.worldmonitor.app in trustedOrigins * fix(auth): add Convex site origins to trustedOrigins * fix(ci): add convex/ to vercel-ignore watched paths * fix(auth): remove admin() plugin — adds banned/role fields rejected by Convex validator * fix(auth): remove admin() plugin — injects banned/role fields rejected by Convex betterAuth validator * feat(auth): replace email/password with email OTP passwordless flow - Replace emailAndPassword + emailVerification with emailOTP plugin - Rewrite AuthModal: email entry -> OTP code verification (no passwords) - Remove admin() plugin (caused Convex schema validation errors) - Remove email verification banner and UNVERIFIED gate reason (OTP inherently verifies email) - Remove password reset flow (forgot/reset password views, token handling) - Clean up unused CSS (tabs, verification banner, success messages) - Update docs to reflect new passwordless auth stack Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(quick-2): harden Convex userRoles and add role cache TTL - P0: Convert setUserRole from mutation to internalMutation (not callable from client) - P2: Add 5-minute TTL to role cache in auth-state.ts - P2: Add localStorage shape warning on auth-token.ts - P3: Document getUserRole public query trade-off - P3: Fix misleading cache comment in auth-session.ts Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(quick-2): auth widget teardown, E2E test rewrite, gateway comment - P2: Store authHeaderWidget on AppContext, destroy in EventHandlerManager.destroy() - P2: Also destroy authModal in destroy() to prevent leaked subscriptions - P1: Rewrite E2E tests for 2-view OTP modal (email input + submit button) - P1: Remove stale "Sign Up" and "Forgot Password" test assertions - P2: Replace flaky waitForTimeout(5000) with Playwright auto-retry assertion - P3: Add clarifying comment on premium bearer-token fallback in gateway Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(header): restructure header/footer, add profile editing, pro-gate playback/export - Remove version, @eliehabib, GitHub link, and download button from header - Move version + @eliehabib credit to footer brand line; download link to footer nav - Move auth widget (profile avatar) to far right of header (after settings gear) - Add default generic SVG avatar for users with no image and no name - Add profile editing in auth dropdown: display name + avatar URL with Save/Cancel - Add Settings shortcut in auth dropdown (opens UnifiedSettings) - Gate Historical Playback and Export controls behind pro role (hidden for free users) - Reactive pro-gate: subscribes to auth state changes, stores unsub in proGateUnsubscribers[] - Clean up proGateUnsubscribers on EventHandlerManager.destroy() to prevent leaks - Fix: render Settings button unconditionally (hidden via style), stable DOM structure - Fix: typed updateUser call with runtime existence check instead of (any) cast - Make initFooterDownload() private to match class conventions * feat(analytics): add Umami auth integration and event tracking - Wire analytics.ts facade to Umami (port from main #1914): search, country, map layers, panels, LLM, theme, language, variant switch, webcam, download, findings, deeplinks - Add Window.umami shim to vite-env.d.ts - Add initAuthAnalytics() that subscribes to auth state and calls identifyUser(id, role) / clearIdentity() on sign-in/sign-out - Add trackSignIn, trackSignUp, trackSignOut, trackGateHit exports - Call initAuthAnalytics() from App.ts after initAuthState() - Track sign-in/sign-up (via isNewUser flag) in AuthModal OTP verify - Track sign-out in AuthHeaderWidget before authClient.signOut() - Track gate-hit for export, playback (event-handlers) and pro-banner * feat(auth): professional avatar widget with colored initials and clean profile edit - Replace white-circle avatar with deterministic colored initials (Gmail/Linear style) - Avatar color derived from email hash across 8-color palette - Dropdown redesigned: row layout with large avatar + name/email/tier info - Profile edit form: name-only (removed avatar URL field) - Remove Settings button from dropdown (gear icon in header is sufficient) - Discord community widget: single CTA link, no redundant text label - Add all missing CSS for dropdown interior, profile edit form, menu items * fix(auth): lock down billing tier visibility and fix TOCTOU race P1: getUserRole converted to internalQuery — billing tier no longer accessible via any public Convex client API. Exposed only through the new authenticated /api/user-role HTTP action which validates the session Bearer token before returning the role. P1: subscribeAuthState generation counter + AbortController prevents rapid sign-in/sign-out from delivering stale role for wrong user. P2: typed RawSessionUser/RawSessionValue interfaces replace any casts at the better-auth nanostore boundary. fetchUserRole drops userId param — server derives identity from Bearer token only. P2: isNewUser heuristic removed from OTP verify — better-auth emailOTP has no reliable isNewUser signal. All verifications tracked as trackSignIn. OTP resend gets 30s client-side cooldown. P2: auth-token.ts version pin comment added (better-auth@1.5.5 + @convex-dev/better-auth@0.11.2). Gateway inner PREMIUM_RPC_PATHS comment clarified to explain why it is not redundant. Adds tests/auth-session.test.mts: 11 tests covering role fallback endpoint selection, fail-closed behavior, and CORS origin matching. * feat(quick-4): replace better-auth with Clerk JS -- packages, Convex config, browser auth layer - Remove better-auth, @convex-dev/better-auth, @better-auth/infra, resend from dependencies - Add @clerk/clerk-js and jose to dependencies - Rewrite convex/auth.config.ts for Clerk issuer domain - Simplify convex/convex.config.ts (remove betterAuth component) - Delete convex/auth.ts, convex/http.ts, convex/userRoles.ts - Remove userRoles table from convex/schema.ts - Create src/services/clerk.ts with Clerk JS init, sign-in, sign-out, token, user metadata, UserButton - Rewrite src/services/auth-state.ts backed by Clerk (same AuthUser/AuthSession interface) - Delete src/services/auth-client.ts (better-auth client) - Delete src/services/auth-token.ts (localStorage token scraping) - Update .env.example with Clerk env vars, remove BETTER_AUTH_API_KEY Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * feat(quick-4): UI components, runtime fetch, server-side JWT, CSP, and tests - Delete AuthModal.ts, create AuthLauncher.ts (thin Clerk.openSignIn wrapper) - Rewrite AuthHeaderWidget.ts to use Clerk UserButton + openSignIn - Update event-handlers.ts to use AuthLauncher instead of AuthModal - Rewrite runtime.ts enrichInitForPremium to use async getClerkToken() - Rewrite server/auth-session.ts for jose-based JWT verification with cached JWKS - Update vercel.json CSP: add *.clerk.accounts.dev to script-src and frame-src - Add Clerk CSP tests to deploy-config.test.mjs - Rewrite e2e/auth-ui.spec.ts for Clerk UI - Rewrite auth-session.test.mts for jose-based validation - Use dynamic import for @clerk/clerk-js to avoid Node.js test breakage Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): allow Clerk Pro users to load premium data on web The data-loader gated premium panel loading (stock-analysis, stock-backtest, daily-market-brief) on WORLDMONITOR_API_KEY only, which is desktop-only. Web users with Clerk Pro auth were seeing unlocked panels stuck on "Loading..." because the requests were never made. Added hasPremiumAccess() helper that checks for EITHER desktop API key OR Clerk Pro role, matching the migration plan Phase 7 requirements. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): address PR #1812 review — all 4 merge blockers + 3 gaps Blockers: 1. Remove stale Convex artifacts (http.js, userRoles.js, betterAuth component) from convex/_generated/api.d.ts 2. isProUser() now checks getAuthState().user?.role === 'pro' alongside legacy localStorage keys 3. Finance premium refresh scheduling now fires for Clerk Pro web users (not just API key holders) 4. JWT verification now validates audience: 'convex' to reject tokens scoped to other Clerk templates Gaps: 5. auth-session tests: 10 new cases (valid pro/free, expired, wrong key/audience/issuer, missing sub/plan, JWKS reuse) using self-signed keys + local JWKS server 6. premium-stock-gateway tests: 4 new bearer token cases (pro→200, free→403, invalid→401, public unaffected) 7. docs/authentication.mdx rewritten for Clerk (removed all better-auth references, updated stack/files/env vars/roles sections) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: address P1 reactive Pro UI + P2 daily-market-brief + P3 stale env vars P1 — In-session Pro UI changes no longer require a full reload: - setupExportPanel: removed early isProUser() return, always creates and relies on reactive subscribeAuthState show/hide - setupPlaybackControl: same pattern — always creates, reactive gate - Custom widget panels: always loaded regardless of Pro status - Pro add-panel and MCP add-panel blocks: always rendered, shown/hidden reactively via subscribeAuthState callback - Flight search wiring: always wired, checks Pro status inside callback so mid-session sign-ins work immediately P2 — daily-market-brief added to hasPremiumAccess() block in loadAllData() so Clerk Pro web users get initial data load (was only primed in primeVisiblePanelData, missing from the general reload path) P3 — Removed stale CONVEX_SITE_URL and VITE_CONVEX_SITE_URL from docs/authentication.mdx env vars table (neither is referenced in codebase) Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: add isProUser import, populate PREMIUM_RPC_PATHS, and fix bearer token auth flow - Added missing isProUser import in App.ts (fixes typecheck) - Populated PREMIUM_RPC_PATHS with stock analysis endpoints - Restructured gateway auth: trusted browser origins bypass API key for premium endpoints (client-side isProUser gate), while bearer token validation runs as a separate step for premium paths when present Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(gateway): require credentials for premium paths + defer free-tier enforcement until auth ready P0: Removed trusted-origin bypass for premium endpoints — Origin header is spoofable and cannot be a security boundary. Premium paths now always require either an API key or valid bearer token. P1: Deferred panel/source free-tier enforcement until auth state resolves. Previously ran in the constructor before initAuthState(), causing Clerk Pro users to have their panels/sources trimmed on every startup. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix(auth): apply WorldMonitor design system to Clerk modal Theme-aware appearance config passed to clerk.load(), openSignIn(), and mountUserButton(). Dark mode: dark bg (#111), green primary (#44ff88), monospace font. Light mode: white bg, green-600 primary (#16a34a). Reads document.documentElement.dataset.theme at call time so theme switches are respected. * fix(auth): gate Clerk init and auth widget behind BETA_MODE Clerk auth initialization and the Sign In header widget are now only activated when localStorage `worldmonitor-beta-mode` is set to "true", allowing silent deployment for internal testing before public rollout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(auth): gate Clerk init and auth widget behind isProUser() Clerk auth initialization and the Sign In header widget are now only activated when the user has wm-widget-key or wm-pro-key in localStorage (i.e. isProUser() returns true), allowing silent deployment for internal testing before public rollout. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(data-loader): replace stale isProUser() with hasPremiumAccess() loadMarketImplications() still referenced the removed isProUser import, causing a TS2304 build error. Align with the rest of data-loader.ts which uses hasPremiumAccess() (checks both API key and Clerk auth). Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(auth): address PR #1812 review — P1 security fixes + P2 improvements P1 fixes: - Add algorithms: ['RS256'] allowlist to jwtVerify (prevents alg:none bypass) - Reset loadPromise on Clerk init failure (allows retry instead of permanent breakage) P2 fixes: - Extract PREMIUM_RPC_PATHS to shared module (eliminates server/client divergence risk) - Add fail-fast guard in convex/auth.config.ts for missing CLERK_JWT_ISSUER_DOMAIN - Add 50s token cache with in-flight dedup to getClerkToken() (prevents concurrent races) - Sync Clerk CSP entries to index.html and tauri.conf.json (previously only in vercel.json) - Type clerkInstance as Clerk instead of any Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * fix(auth): clear cached token on signOut() Prevents stale token from being returned during the ≤50s cache window after a user signs out. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> --------- Co-authored-by: Sebastien Melki <sebastien@anghami.com> Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> Co-authored-by: Sebastien Melki <sebastienmelki@gmail.com> |
||
|
Elie Habib
|
2939b1f4a1 |
feat(finance-panels): add 7 macro/market panels + Daily Brief context (issues #2245-#2253) (#2258)
* feat(fear-greed): add regime state label, action stance badge, divergence warnings Closes #2245 * feat(finance-panels): add 7 new finance panels + Daily Brief macro context Implements issues #2245 (F&G Regime), #2246 (Sector Heatmap bars), #2247 (MacroTiles), #2248 (FSI), #2249 (Yield Curve), #2250 (Earnings Calendar), #2251 (Economic Calendar), #2252 (COT Positioning), #2253 (Daily Brief prompt extension). New panels: - MacroTilesPanel: CPI YoY, Unemployment, GDP, Fed Rate tiles via FRED - FSIPanel: Financial Stress Indicator gauge (HYG/TLT/VIX/HY-spread) - YieldCurvePanel: SVG yield curve chart with inverted/normal badge - EarningsCalendarPanel: Finnhub earnings calendar with BMO/AMC/BEAT/MISS - EconomicCalendarPanel: FOMC/CPI/NFP events with impact badges - CotPositioningPanel: CFTC disaggregated COT positioning bars - MarketPanel: adds sorted bar chart view above sector heatmap grid New RPCs: - ListEarningsCalendar (market/v1) - GetCotPositioning (market/v1) - GetEconomicCalendar (economic/v1) Seed scripts: - seed-earnings-calendar.mjs (Finnhub, 14-day window, TTL 12h) - seed-economic-calendar.mjs (Finnhub, 30-day window, TTL 12h) - seed-cot.mjs (CFTC disaggregated text file, TTL 7d) - seed-economy.mjs: adds yield curve tenors DGS1MO/3MO/6MO/1/2/5/30 - seed-fear-greed.mjs: adds FSI computation + sector performance Daily Brief: extends buildDailyMarketBrief with optional regime, yield curve, and sector context fed to the LLM summarization prompt. All panels default enabled in FINANCE_PANELS, disabled in FULL_PANELS. 🤖 Generated with Claude Sonnet 4.6 via Claude Code (https://claude.ai/claude-code) + Compound Engineering v2.40.0 Co-Authored-By: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com> * fix(finance-panels): address code review P1/P2 findings P1 - Security/Correctness: - EconomicCalendarPanel: add escapeHtml on all 7 Finnhub-sourced fields - EconomicCalendarPanel: fix panel contract (public fetchData():boolean, remove constructor self-init, add retry callbacks to all showError calls) - YieldCurvePanel: fix NaN in xPos() when count <= 1 (divide-by-zero) - seed-earnings-calendar: move Finnhub API key from URL to X-Finnhub-Token header - seed-economic-calendar: move Finnhub API key from URL to X-Finnhub-Token header - seed-earnings-calendar: add isMain guard around runSeed() call - health.js + bootstrap.js: register earningsCalendar, econCalendar, cotPositioning keys - health.js dataSize(): add earnings + instruments to property name list P2 - Quality: - FSIPanel: change !resp.fsiValue → resp.fsiValue <= 0 (rejects valid zero) - data-loader: fix Promise.allSettled type inference via indexed destructure - seed-fear-greed: allowlist cnnLabel against known values before writing to Redis - seed-economic-calendar: remove unused sleep import - seed-earnings-calendar + econ-calendar: increase TTL 43200 → 129600 (36h = 3x interval) - YieldCurvePanel: use SERIES_IDS const in RPC call (single source of truth) * fix(bootstrap): remove on-demand panel keys from bootstrap.js earningsCalendar, econCalendar, cotPositioning panels fetch via RPC on demand — they have no getHydratedData consumer in src/ and must not be in api/bootstrap.js. They remain in api/health.js BOOTSTRAP_KEYS for staleness monitoring. * fix(compound-engineering): fix markdown lint error in local settings * fix(finance-panels): resolve all P3 code-review findings - 030: MacroTilesPanel: add `deltaFormat?` field to MacroTile interface, define per-tile delta formatters (CPI pp, GDP localeString+B), replace fragile tile.id switch in tileHtml with fmt = deltaFormat ?? format - 031: FSIPanel: check getHydratedData('fearGreedIndex') at top of fetchData(); extract fsi/vix/hySpread from headerMetrics and render synchronously; fall back to live RPC only when bootstrap absent - 032: All 6 finance panels: extract lazy module-level client singletons (EconomicServiceClient or MarketServiceClient) so the client is constructed at most once per panel module lifetime, not on every fetchData - 033: get-fred-series-batch: add BAMLC0A0CM and SOFR to ALLOWED_SERIES (both seeded by seed-economy.mjs but previously unreachable via RPC) * fix(finance-panels): health.js SEED_META, FSI calibration, seed-cot catch handler - health.js: add SEED_META entries for earningsCalendar (1440min), econCalendar (1440min), cotPositioning (14400min) — without these, stopped seeds only alarm CRIT:EMPTY after TTL expiry instead of earlier WARN:STALE_SEED - seed-cot.mjs: replace bare await with .catch() handler consistent with other seeds - seed-fear-greed.mjs: recalibrate FSI thresholds to match formula output range (Low>=1.5, Moderate>=0.8, Elevated>=0.3; old values >=0.08/0.05/0.03 were calibrated for [0,0.15] but formula yields ~1-2 in normal conditions) - FSIPanel.ts: fix gauge fillPct range to [0, 2.5] matching recalibrated thresholds - todos: fix MD022/MD032 markdown lint errors in P3 review files --------- Co-authored-by: Claude Sonnet 4.6 (200K context) <noreply@anthropic.com> |
||
|
Elie Habib
|
45469fae3b |
feat(forecasts): NEXUS panel redesign + simulation theater data via theaterSummariesJson (#2244)
Redesigns ForecastPanel with a theater-first NEXUS layout that surfaces simulation outcomes alongside forecast probabilities in a unified view. Adds the theaterSummariesJson field to the GetSimulationOutcome proto so the server can return pre-condensed UI data from a single Redis read without any additional R2 fetches. - proto: add theater_summaries_json = 9 to GetSimulationOutcomeResponse - seed-forecasts.mjs: embed condensed uiTheaters array in Redis pointer at write time - get-simulation-outcome.ts: serialize pointer.uiTheaters → theaterSummariesJson in RPC response - src/services/forecast.ts: add fetchSimulationOutcome() returning theaterSummariesJson string - src/app/data-loader.ts: load simulation outcome alongside forecasts, call updateSimulation() - ForecastPanel.ts: full NEXUS redesign with SVG circular gauges, expandable theater detail, compact prob table, CSS custom property for per-theater accent color, race condition guard (skip render when forecasts array still empty on simulation arrival) |
||
|
Elie Habib
|
54eda4b54b |
fix(bls): replace BLS API with FRED to unblock Labor tab in Macro Stress panel (#2238)
api.bls.gov rejects HTTPS CONNECT tunnels from Railway container IPs even through Decodo proxy (gate.decodo.com returns 403 on CONNECT for .gov domains). FRED mirrors the national BLS series with identical data and no IP restrictions. - seed-bls-series.mjs: rewrite to fetch USPRIV + ECIALLCIV from FRED (api.stlouisfed.org works through Railway, confirmed by existing seed-economy). Converts FRED date format to BLS observation shape (year/period/periodName/value) so the existing handler and frontend parsing are unchanged. Metro-area LAUMT* series dropped — no FRED equivalent available. - get-bls-series.ts: update KNOWN_SERIES_IDS allowlist to USPRIV + ECIALLCIV - economic/index.ts: update BLS_SERIES IDs and clear BLS_METRO_IDS This unblocks the Labor tab in the Macro Stress panel which has shown blank since the seeder was deployed (every BLS fetch = timeout/fetch failed). |
||
|
Elie Habib
|
01f6057389 |
feat(simulation): MiroFish Phase 2 — theater-limited simulation runner (#2220)
* feat(simulation): MiroFish Phase 2 — theater-limited simulation runner Adds the simulation execution layer that consumes simulation-package.json and produces simulation-outcome.json for maritime chokepoint + energy/logistics theaters, closing the WorldMonitor → MiroFish handoff loop. Changes: - scripts/seed-forecasts.mjs: 2-round LLM simulation runner (prompt builders, JSON extractor, runTheaterSimulation, writeSimulationOutcome, task queue with NX dedup lock, runSimulationWorker poll loop) - scripts/process-simulation-tasks.mjs: standalone worker entry point - proto: GetSimulationOutcome RPC + make generate - server/worldmonitor/forecast/v1/get-simulation-outcome.ts: RPC handler - server/gateway.ts: slow tier for get-simulation-outcome - api/health.js: simulationOutcomeLatest in STANDALONE + ON_DEMAND keys - tests: 14 new tests for simulation runner functions * fix(simulation): address P1/P2 code review findings from PR #2220 Security (P1 #018): - sanitizeForPrompt() applied to all entity/seed fields interpolated into Round 1 prompt (entityId, class, stance, seedId, type, timing) - sanitizeForPrompt() applied to actorId and entityIds in Round 2 prompt - sanitizeForPrompt() + length caps applied to all LLM array fields written to R2 (dominantReactions, stabilizers, invalidators, keyActors, timingMarkers) Validation (P1 #019): - Added validateRunId() regex guard - Applied in enqueueSimulationTask() and processNextSimulationTask() loop Type safety (P1 #020): - Added isOutcomePointer() and isPackagePointer() type guards in TS handlers - Replaced unsafe as-casts with runtime-validated guards in both handlers Correctness (P2 #022): - Log warning when pkgPointer.runId does not match task runId Architecture (P2 #024): - isMaritimeChokeEnergyCandidate() accepts both flat and nested topBucketId - Call site simplified to pass theater directly Performance (P2 #025): - SIMULATION_ROUND1_MAX_TOKENS raised 1800 to 2200 - Added max 3 initialReactions instruction to Round 1 prompt Maintainability (P2 #026): - Simulation pointer keys exported from server/_shared/cache-keys.ts - Both TS handlers import from shared location Documentation (P2 #027): - Strengthened runId no-op description in proto and OpenAPI spec * fix(todos): add blank lines around lists in markdown todo files * style(api): reformat openapi yaml to match linter output * test(simulation): add flat-shape filter test + getSimulationOutcome handler coverage Two tests identified as missing during PR #2220 review: 1. isMaritimeChokeEnergyCandidate flat-shape tests — covers the || candidate.topBucketId normalization added in the P1/P2 review pass. The existing tests only used the nested marketContext.topBucketId shape; this adds the flat root-field shape that arrives from the simulation-package.json JSON (selectedTheaters entries have topBucketId at root). 2. getSimulationOutcome handler structural tests — verifies the isOutcomePointer guard, found:false NOT_FOUND return, found:true success path, note population on runId mismatch, and redis_unavailable error string. Follows the readSrc static-analysis pattern used elsewhere in server-handlers.test.mjs (handler imports Redis so full integration test would require a test Redis instance). |
||
|
Elie Habib
|
e548e6cca5 |
feat(intelligence): cross-source signal aggregator with composite escalation (#2143) (#2164)
* feat(intelligence): cross-source signal aggregator with composite escalation (#2143) Adds a threshold-based signal aggregator seeder that reads 15+ already-seeded Redis keys every 15 minutes, ranks cross-domain signals by severity, and detects composite escalation when >=3 signal categories co-fire in the same theater. * fix(cross-source-signals): wire panel data loading, inline styles, seeder cleanup - New src/services/cross-source-signals.ts: fetch via IntelligenceServiceClient with circuit breaker - data-loader.ts: add loadCrossSourceSignals() + startup batch entry (SITE_VARIANT !== 'happy' guard) - App.ts: add primeVisiblePanelData entry + scheduleRefresh at 15min interval - base.ts: add crossSourceSignals: 15 * 60 * 1000 to REFRESH_INTERVALS - CrossSourceSignalsPanel.ts: replace all CSS class usage with inline styles (MarketImplicationsPanel pattern) - seed-cross-source-signals.mjs: remove dead isMain var, fix afterPublish double-write, deterministic signal IDs, GDELT per-topic tone keys (military/nuclear/maritime) with 3-point declining trend + < -1.5 threshold per spec, bundled topics fallback * fix(cross-source-signals): complete bootstrap wiring, seeder fixes, cmd-k entry - cache-keys.ts: add crossSourceSignals to BOOTSTRAP_CACHE_KEYS + BOOTSTRAP_TIERS (slow) - bootstrap.js: add crossSourceSignals key + SLOW_KEYS entry - cross-source-signals.ts: add getHydratedData('crossSourceSignals') bootstrap hydration - seed script: fix isDeclinig typo, maritime theater ternary (Global->Indo-Pacific), displacement year dynamic - commands.ts: add panel:cross-source-signals to cmd-k * feat(cross-source-signals): redesign panel — severity bars, filled badges, icons, theater pills - 4px severity accent bar on all signal rows (scannable without reading badges) - Filled severity badges: CRITICAL=solid red/white, HIGH=faint red bg, MED=faint yellow bg - Type badge emoji prefix: ⚡ composite, 🔴 geo-physical, 📡 EW, ✈️ military, 📊 market, ⚠️ geopolitical - Composite card: full glow (box-shadow) instead of 3px left border only - Theater pill with inline age: "Middle East · 8m ago" - Contributor pills: individual chips instead of dot-separated string - Pulsing dot on composite escalation banner * fix(cross-source-signals): code review fixes — module-level Sets, signal cap, keyframe scoping, OREF expansion - Replace per-call Array literals in list-cross-source-signals.ts with module-level Set constants for O(1) lookups - Add index-based fallback ID in normalizeSignal to avoid undefined ids - Remove unused military:flights:stale:v1 from SOURCE_KEYS - Add MAX_SIGNALS=30 cap before writing to Redis - Expand extractOrefAlertCluster to any "do not travel" advisory (not just Israel) - Add BASE_WEIGHT inline documentation explaining scoring scale - Fix animation keyframe: move from setContent() <style> block to constructor (injected once), rename to cross-source-pulse-dot - Fix GDELT extractor to read per-topic gdelt:intel:tone:{topic} keys with correct decline logic - Fix isDeclinig typo, maritime dead ternary, and displacement year reference |
||
|
Elie Habib
|
f87c8c71c4 |
feat(forecast): Phase 2 simulation package read path (#2219)
* feat(forecast): Phase 2 simulation package read path (getSimulationPackage RPC + Redis existence key)
- writeSimulationPackage now writes forecast:simulation-package:latest to Redis after
successful R2 write, containing { runId, pkgKey, schemaVersion, theaterCount, generatedAt }
with TTL matching TRACE_REDIS_TTL_SECONDS (60 days)
- New getSimulationPackage RPC handler reads Redis key, returns pointer metadata without
requiring an R2 fetch (zero R2 cost for existence check)
- Wired into ForecastServiceHandler and server/gateway.ts cache tier (medium)
- Proto: GetSimulationPackage RPC + get_simulation_package.proto message definitions
- api/health.js: simulationPackageLatest added to STANDALONE_KEYS + ON_DEMAND_KEYS
- Tests: SIMULATION_PACKAGE_LATEST_KEY constant + writeSimulationPackage null-guard test
Closes todo #017 (Phase 2 prerequisites for MiroFish integration)
* chore(generated): regenerate proto types for GetSimulationPackage RPC
* fix(simulation-rpc): distinguish Redis failure from not-found; signal runId mismatch
- Add `error` field to GetSimulationPackageResponse: populated with
"redis_unavailable" on Redis errors so callers can distinguish a
healthy not-found (found=false, error="") from a Redis failure
(found=false, error="redis_unavailable"). Adds console.warn on error.
- Add `note` field: populated when req.runId is supplied but does not
match the latest package's runId, signalling that per-run filtering
is not yet active (Phase 3).
- Add proto comment on run_id: "Currently ignored; reserved for Phase 3"
- Add milliseconds annotation to generated_at description.
- Simplify handler: extract NOT_FOUND constant, remove SimulationPackagePointer
interface, remove || '' / || 0 guards on guaranteed-present fields.
- Regenerate all buf-generated files.
Fixes todos #018 (runId silently ignored) and #019 (error indistinguishable
from not-found). Also resolves todos #022 (simplifications) and #023
(OpenAPI required fields / generatedAt unit annotation).
* fix(simulation-rpc): change cache tier from medium to slow (aligns with deep-run update frequency)
* fix(simulation-rpc): fix key prefixing, make Redis errors reachable, no-cache not-found
Three P1 regressions caught in external review:
1. Key prefix bug: getCachedJson() applies preview:<sha>: prefix in non-production
environments, but writeSimulationPackage writes the raw key via a direct Redis
command. In preview/dev the RPC always returned found:false even when the package
existed. Fix: new getRawJson() in redis.ts always uses the unprefixed key AND throws
on failure instead of swallowing errors.
2. redis_unavailable unreachable: getCachedJson swallows fetch failures and missing-
credentials by returning null, so the catch block for redis_unavailable was dead
code. getRawJson() throws on HTTP errors and missing credentials, making the
error: "redis_unavailable" contract actually reachable.
3. Negative-cache stampede: slow tier caches every 200 GET. A request before any deep
run wrote a package returned { found:false } which the CDN cached for up to 1h,
breaking post-run discovery. Fix: markNoCacheResponse() on both not-found and
error paths so they are served fresh on every request.
|
||
|
Elie Habib
|
7013b2f9f1 |
feat(market): Fear & Greed Index 2.0 — 10-category composite sentiment panel (#2181)
* Add Fear & Greed Index 2.0 reverse engineering brief Analyzes the 10-category weighted composite (Sentiment, Volatility, Positioning, Trend, Breadth, Momentum, Liquidity, Credit, Macro, Cross-Asset) with scoring formulas, data source audit, and implementation plan for building it as a worldmonitor panel. https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i * Add seed script implementation plan to F&G brief Details exact endpoints, Yahoo symbols (17 calls), Redis key schema, computed metrics, FRED series to add (BAMLC0A0CM, SOFR), CNN/AAII sources, output JSON schema, and estimated runtime (~8s per seed run). https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i * Update brief: all sources are free, zero paid APIs needed - CBOE CDN CSVs for put/call ratios (totalpc.csv, equitypc.csv) - CNN dataviz API for Fear & Greed (production.dataviz.cnn.io) - Yahoo Finance for VIX9D/VIX3M/SKEW/RSP/NYA (standard symbols) - FRED for IG spread (BAMLC0A0CM) and SOFR (add to existing array) - AAII scrape for bull/bear survey (only medium-effort source) - Breadth via RSP/SPY divergence + NYSE composite (no scraping) https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i * Add verified Yahoo symbols for breadth + finalized source list New discoveries: - ^MMTH = % stocks above 200 DMA (direct Yahoo symbol!) - C:ISSU = NYSE advance/decline data - CNN endpoint accepts date param for historical data - CBOE CSVs have data back to 2003 - 33 total calls per seed run, ~6s runtime All 10 categories now have confirmed free sources. https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i * Rewrite F&G brief as forward-looking design doc Remove all reverse-engineering language, screenshot references, and discovery notes. Clean structure: goal, scoring model, data sources, formulas, seed script plan, implementation phases, MVP path. https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i * docs: apply gold standard corrections to fear-greed-index-2.0 brief * feat(market): add Fear & Greed Index 2.0 — 10-category composite sentiment panel Composite 0-100 index from 10 weighted categories: sentiment (CNN F&G, AAII, crypto F&G), volatility (VIX, term structure), positioning (P/C ratio, SKEW), trend (SPX vs MAs), breadth (% >200d, RSP/SPY divergence), momentum (sector RSI, ROC), liquidity (M2, Fed BS, SOFR), credit (HY/IG spreads), macro (Fed rate, yield curve, unemployment), cross-asset (gold/bonds/DXY vs equities). Data layer: - seed-fear-greed.mjs: 19 Yahoo symbols (150ms gaps), CBOE P/C CSVs, CNN F&G API, AAII scrape (degraded-safe), FRED Redis reads. TTL 64800s. - seed-economy.mjs: add BAMLC0A0CM (IG spread) and SOFR to FRED_SERIES. - Bootstrap 4-file checklist: cache-keys, bootstrap.js, health.js, handler. Proto + RPC: - get_fear_greed_index.proto with FearGreedCategory message. - get-fear-greed-index.ts handler reads seeded Redis data. Frontend: - FearGreedPanel with gauge, 9-metric header grid, 10-category breakdown. - Self-loading via bootstrap hydration + RPC fallback. - Registered in panel-layout, App.ts (prime + refresh), panel config, Cmd-K commands, finance variant, i18n (en/ar/zh/es). * fix(market): add RPC_CACHE_TIER entry for get-fear-greed-index * fix(docs): escape bare angle bracket in fear-greed brief for MDX * fix(docs): fix markdown lint errors in fear-greed brief (blank lines around headings/lists) * fix(market): fix seed-fear-greed bugs from code review - fredLatest/fredNMonthsAgo: guard parseFloat with Number.isFinite to handle FRED's "." missing-data sentinel (was returning NaN which propagated through scoring as a truthy non-null value) - Remove 3 unused Yahoo symbols (^NYA, HYG, LQD) that were fetched but not referenced in any scoring category (saves ~450ms per run) - fedRateStr: display effective rate directly instead of deriving target range via (fedRate - 0.25) which was incorrect * fix(market): address P2/P3 review findings in Fear & Greed - FearGreedPanel: add mapSeedPayload() to correctly map raw seed JSON to proto-shaped FearGreedData; bootstrap hydration was always falling through to RPC because seed shape (composite.score) differs from proto shape (compositeScore) - FearGreedPanel: fix fmt() — remove === 0 guard and add explicit > 0 checks on VIX and P/C Ratio display to handle proto default zeros without masking genuine zero values (e.g. pctAbove200d) - seed-fear-greed: remove broken history write — each run overwrote the key with a single-entry array (no read-then-append), making the 90-day TTL meaningless; no consumer exists yet so defer to later - seed-fear-greed: extract hySpreadVal const to avoid double fredLatest call - seed-fear-greed: fix stale comment (19 symbols → 16 after prior cleanup) --------- Co-authored-by: Claude <noreply@anthropic.com> |
||
|
Elie Habib
|
a1c3c1d684 |
feat(panels): AI Market Implications — LLM trade signals from live world state (#2146) (#2165)
* fix(intelligence): use camelCase field names for ListMarketImplicationsResponse
* fix(bootstrap): register marketImplications in cache-keys.ts and add hydration consumer
* chore: stage all market-implications feature files for proto freshness check
* feat(market-implications): add LLM routing env vars for market implications stage
* fix(market-implications): move types to services layer to fix boundary violation
* fix: add list-market-implications gateway tier entry
* fix(market-implications): add health.js entries + i18n tooltip key
- api/health.js: add marketImplications to BOOTSTRAP_KEYS
('intelligence:market-implications:v1') and SEED_META
(seed-meta:intelligence:market-implications, maxStaleMin=150 = 2x
the 75min TTL, matching gold standard)
- en.json: add components.marketImplications.infoTooltip which was
referenced in MarketImplicationsPanel but missing from locales
* fix(market-implications): wire CMD+K entry and panels.marketImplications i18n key
- commands.ts: add panel:market-implications command with trade/signal
keywords so the panel appears in CMD+K search
- en.json: add panels.marketImplications used by UnifiedSettings panel
toggle display and SearchModal label resolution
|
||
|
Elie Habib
|
bbffbd6998 |
feat(economic): BLS direct integration for CES/LAUMT/ECI series (#2141)
* feat(economic): BLS direct integration for CES/LAUMT/ECI series (#2046) * fix(bls-series): add isMain guard, series allowlist, and empty-value filter - Wrap runSeed() in isMain guard (process.argv[1]) to prevent seed from executing when the module is imported by tests or other scripts. This is the critical codebase-wide pattern documented in MEMORY.md. - Add KNOWN_SERIES_IDS allowlist in getBlsSeries handler to block arbitrary Redis key enumeration via user-supplied series_id values. - Fix observation filter in seed script: add d.value truthiness check so empty strings from null/undefined BLS values do not pass through as valid observations alongside the existing dash-value guard. * fix(bls-series): align TTL, cache tier, and maxStaleMin with gold standard - CACHE_TTL: 86400 → 259200 (3× daily interval) - maxStaleMin: 1440 → 2880 (2× daily interval) - gateway cache tier: static → daily (CDN s-maxage=86400 for daily-seeded data) - process.exit(1) → process.exit(0) * feat(economic): surface BLS series in EconomicPanel Labor Market tab Adds Labor Market tab to EconomicPanel (TODO-088) consuming the BLS direct integration from PR #2141. Closes issue #2046 UI gap. - fetchBlsData() in economic service: parallel getBlsSeries calls for all 5 series, adapted to FredSeries shape with sparkline data - EconomicPanel: Labor Market tab with national section (payrolls, ECI) and Metro Unemployment sub-section (SF, Boston, NYC) - Tab hidden gracefully when BLS data unavailable (cron not yet run) - loadBlsData() wired in data-loader parallel task list - DataSourceId + data-freshness metadata for bls source - All 21 locales: laborMarket + metroUnemployment keys * fix(generated): restore main-compatible generated files with BLS additions * fix(generated): correct indentation from manual merge |
||
|
Elie Habib
|
e08457aadf |
feat(fuel-prices): add retail gasoline and diesel prices panel (#2150)
* feat(fuel-prices): add retail fuel prices panel and seeder - Add ListFuelPrices proto RPC with FuelPrice/FuelCountryPrice messages - Create seed-fuel-prices.mjs seeder with 5 sources: Malaysia (data.gov.my), Spain (minetur.gob.es), Mexico (datos.gob.mx), US EIA, EU oil bulletin CSV - Add list-fuel-prices.ts RPC handler reading from Redis seed cache - Wire handler into EconomicService handler.ts - Register fuelPrices in cache-keys.ts, bootstrap.js, health.js, gateway.ts - Add FuelPricesPanel frontend component with gasoline/diesel/source columns - Wire panel into panel-layout.ts, App.ts (prime + refresh scheduler) - Add panel config, command entry with fuel/gas/diesel/petrol keywords - Add fuelPrices refresh interval (6h) in base.ts - Add i18n keys in en.json (panels.fuelPrices + components.fuelPrices) Task: fuel-prices * fix(fuel-prices): add BR/NZ/UK sources, fix EU non-euro currency, review fixes - Add fetchBrazil() — ANP CSVs (GASOLINA + DIESEL), Promise.allSettled for independent partial results, BRL→USD via FX - Add fetchNewZealand() — MBIE weekly-table.csv, Board price national avg, NZD→USD - Add fetchUK_ModeA() — CMA retailer JSON feeds (Asda/BP/JET/MFG/Sainsbury's/ Morrisons), E10+B7 pence→GBP, max-date observedAt across retailers - Fix EU non-euro members (BG/CZ/DK/HU/PL/RO/SE) using local currency FX on EUR-denominated prices — all EU entries now use currency:'EUR' - Fix fetchBrazil Promise.all → Promise.allSettled (partial CSV failure no longer discards both fuels) - Fix UK observedAt: keep latest date across retailers (not last-processed) - Fix WoW anomaly: omit wowPct instead of setting to 0 - Lift parseEUPrice out of inner loop to module scope - Pre-compute parseBRDate per row to avoid double-conversion - Update infoTooltip: describe methodology without exposing source URLs - Add BRL, NZD, GBP to FX symbols list * fix(fuel-prices): fix 4 live data bugs found in external review EU CSV: replace hardcoded 2024 URLs (both returning 404) with dynamic discovery — scrape EC energy page for current CSV link, fall back to generated YYYY-MM patterns for last 4 months. NZ: live MBIE header has no Region column (Week,Date,Fuel,Variable,Value, Unit,Status) — remove regionIdx guard that was forcing return []. Values are in NZD c/L not NZD/L — divide by 100 before storing. UK: last_updated is DD/MM/YYYY HH:mm:ss not ISO — parse to YYYY-MM-DD before lexicographic max-date comparison; previous code stored the seed-run date instead of the latest retailer timestamp. Panel: source column fell back to — for diesel-only countries because it only read gas?.source. Use (gas ?? dsl)?.source so diesel-only rows display their source correctly. |
||
|
Elie Habib
|
4f19e36804 |
feat(intelligence): GDELT tone/vol timeline analysis for escalation signals (#2044) (#2087)
* feat(intelligence): GDELT tone/vol timeline per topic (#2044) * fix(gdelt-timeline): add isMain guard to seed script, fix gateway cache tier - Wrap runSeed() call in isMain guard (process.argv[1].endsWith check) to prevent CI failures when the seed module is imported rather than executed directly — pre-push hook does not catch this - Change gateway cache tier from 'medium' (20min CDN) to 'daily' (1h browser/s-maxage=86400 CDN) to align with the 1h TIMELINE_TTL on the per-topic tone/vol Redis keys * fix(gdelt-timeline): TTL 1h→12h, medium cache tier, real fetchedAt, exit 0 - seed-gdelt-intel.mjs: TIMELINE_TTL 3600→43200 (12h = 2× 6h cron) so tone/vol keys survive between cron runs instead of expiring after 1h - seed-gdelt-intel.mjs: afterPublish wraps tone/vol as {data, fetchedAt} so the real write timestamp is stored alongside the arrays - get-gdelt-topic-timeline.ts: unwrap new envelope shape; fetchedAt now reflects actual data write time instead of request time - gateway.ts: daily→medium cache tier (CDN s-maxage=1200 matches 6h cadence) - seed-gdelt-intel.mjs: process.exit(1)→0 to match seeder suite convention * fix(gdelt-timeline): add GdeltTimelinePoint type cast in unwrap helper |
||
|
Elie Habib
|
9696a545eb |
feat(trade): UN Comtrade strategic commodity flows seeder + RPC (#2045) (#2089)
* feat(trade): UN Comtrade strategic commodity flows seeder + RPC (#2045) * fix(trade): correct byYear overwrite bug and anomaliesOnly upstream flag Two bugs in the Comtrade flows feature: 1. seed-trade-flows.mjs: byYear Map used year alone as key. With flowCode=X,M the API returns both export and import records for the same year; the second record silently overwrote the first, causing incorrect val/wt and YoY calculations. Fix: key by `${flowCode}:${year}` so exports and imports are tracked separately and YoY is computed per flow direction. 2. list-comtrade-flows.ts: `if (!flows.length)` set upstreamUnavailable=true even when Redis data was present but all records were filtered out by anomaliesOnly=true. Fix: track dataFound separately and only set upstreamUnavailable when no Redis keys returned data. * fix(comtrade-flows): gold standard TTL, maxStaleMin, exit code, batch Redis fetch - health.js: maxStaleMin 1440→2880 (2× daily interval per gold standard) - seed-trade-flows.mjs: CACHE_TTL 86400→259200 (72h = 3× daily interval) - seed-trade-flows.mjs: process.exit(1)→0 to match seeder suite convention - list-comtrade-flows.ts: replace 30 getCachedJson calls with single getCachedJsonBatch pipeline |
||
|
Elie Habib
|
3321069fb3 |
feat(sanctions): entity lookup index + OpenSanctions search (#2042) (#2085)
* feat(sanctions): entity lookup index + OpenSanctions search (#2042) * fix: guard tokens[0] access in sanctions lookup * fix: use createIpRateLimiter pattern in sanctions-entity-search * fix: add sanctions-entity-search to allowlist and cache tier * fix: add LookupSanctionEntity RPC to service.proto, regenerate * fix(sanctions): strip _entityIndex/_state from main key publish, guard limit NaN P0: seed-sanctions-pressure was writing the full _entityIndex array and _state snapshot into sanctions:pressure:v1 because afterPublish runs after atomicPublish. Add publishTransform to strip both fields before the main key write so the pressure payload stays compact; afterPublish and extraKeys still receive the full data object and write the correct separate keys. P1: limit param in sanctions-entity-search edge function passed NaN to OpenSanctions when a non-numeric value was supplied. Fix with Number.isFinite guard. P2: add 200-char max length on q param to prevent oversized upstream requests. * fix(sanctions): maxStaleMin 2x interval, no-store on entity search health.js: 720min (1x) → 1440min (2x) for both sanctionsPressure and sanctionsEntities. A single missed 12h cron was immediately flagging stale. sanctions-entity-search.js: Cache-Control public → no-store. Sanctions lookups include compliance-sensitive names in the query string; public caching would have logged/stored these at CDN/proxy layer. |
||
|
Elie Habib
|
ed358c0cb0 |
feat(cii): wire news:insights:v1 threat levels into newsActivity scoring (#2095)
* feat(intelligence): add countryCode geo-attribution to topStories (#2051) * fix(geo-extract): filter EU as supranational, add unigram stopwords, type countryCode in ServerInsightStory - Map 'eu'/'europe' to 'XX' (supranational marker, returns null) instead of 'EU' which is not a valid ISO2 code and would be silently ignored by downstream CII scorer - Add UNIGRAM_STOPWORDS set for high-false-positive single-word entries in country-names.json: chad/jordan/georgia/niger/guinea/mali/peru — these match too frequently as person names and US state names in English headlines; their country meanings are covered by unambiguous aliases (nigerian, georgian context via bigrams, etc.) - Add countryCode: string | null and pubDate: string to ServerInsightStory TypeScript interface to match what seed-insights.mjs now writes to Redis * fix(geo-extract): add 'us' to UNIGRAM_STOPWORDS to prevent pronoun false positives 'us' as a bare word matches almost every English headline ("give us", "tells us", etc.). US coverage is preserved via the 'washington' and 'american' aliases in ALIAS_MAP. * fix(geo-extract): fix US abbreviation, bigram punctuation, and scan ordering Three issues: 1. 'us' stopword suppressed uppercase US (country). Fix: pre-process \bUS\b → 'United States' before lowercasing; remove 'us' from stopwords. 2. Bigram matching used raw tokens so 'West Bank,' and 'Tel Aviv:' missed their alias entries. Fix: strip punctuation from each token before forming the bigram key. 3. Two-pass scan (all bigrams then all unigrams) meant 'United States' bigram fired before earlier unigrams like 'Iran' in 'Iran blames US'. Fix: single left-to-right scan with local longest-match (bigram at i before unigram at i), preserving first-mention document order. * feat(cii): wire news:insights:v1 threat levels into CII newsActivity scoring Reads news:insights:v1 alongside other auxiliary sources in fetchAuxiliarySources(). Per-story threat weights (critical→4, high→2, medium/elevated→1, low→0.5, info→0) are accumulated per country using countryCode (geo-extracted in #2051) with COUNTRY_KEYWORDS title fallback, capped at 20. Replaces the hardcoded information=0 in computeCIIScores(). Closes #2052 * fix(cii): resolve TS errors in news scoring (undefined narrowing, emptyAux) * test(cii): add newsTopStories to emptyAux test fixture * fix(cii): add moderate threat weight and expand newsScore test coverage - Add 'moderate' to THREAT_WEIGHT (0.5) so seed-insights.mjs stories classified as moderate/political/economic are not silently discarded - Change fetchAuxiliarySources threatLevel fallback from 'moderate' to 'low' (consistent with unknown-importance stories) - Add 5 tests: critical attribution, cap-at-20, moderate handling, null-countryCode keyword fallback, info-level no-op * chore: add docs/ideation/ to .gitignore (private ideation notes) |
||
|
Elie Habib
|
0ef2bbe38d | chore(intelligence): remove dead _batch-classify.ts helper (#2049) (#2092) | ||
|
Elie Habib
|
eaf4c771ad |
feat(market+economy): add Russell 2000 index and GSCPI seeder (#2140)
Russell 2000 (^RUT): - Add to MARKET_SYMBOLS and YAHOO_ONLY in ais-relay.cjs - Covers the 4th major US index missing from the panel GSCPI (NY Fed Global Supply Chain Pressure Index): - New seedGscpi() loop in ais-relay.cjs — fetches monthly CSV from newyorkfed.org (no API key required), parses wide-format vintage CSV - Stored as economic:fred:v1:GSCPI:0 in FRED-compatible format so the existing GetFredSeriesBatch RPC serves it without any proto changes - TTL 72h (3x 24h interval), retry 20min on failure — gold standard pattern - Add GSCPI to ALLOWED_SERIES in get-fred-series-batch.ts - Add gscpi to STANDALONE_KEYS + SEED_META in health.js (maxStaleMin 2880) |