Files
worldmonitor/docs/fear-greed-index-2.0-brief.md
Elie Habib 7013b2f9f1 feat(market): Fear & Greed Index 2.0 — 10-category composite sentiment panel (#2181)
* Add Fear & Greed Index 2.0 reverse engineering brief

Analyzes the 10-category weighted composite (Sentiment, Volatility,
Positioning, Trend, Breadth, Momentum, Liquidity, Credit, Macro,
Cross-Asset) with scoring formulas, data source audit, and
implementation plan for building it as a worldmonitor panel.

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* Add seed script implementation plan to F&G brief

Details exact endpoints, Yahoo symbols (17 calls), Redis key schema,
computed metrics, FRED series to add (BAMLC0A0CM, SOFR), CNN/AAII
sources, output JSON schema, and estimated runtime (~8s per seed run).

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* Update brief: all sources are free, zero paid APIs needed

- CBOE CDN CSVs for put/call ratios (totalpc.csv, equitypc.csv)
- CNN dataviz API for Fear & Greed (production.dataviz.cnn.io)
- Yahoo Finance for VIX9D/VIX3M/SKEW/RSP/NYA (standard symbols)
- FRED for IG spread (BAMLC0A0CM) and SOFR (add to existing array)
- AAII scrape for bull/bear survey (only medium-effort source)
- Breadth via RSP/SPY divergence + NYSE composite (no scraping)

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* Add verified Yahoo symbols for breadth + finalized source list

New discoveries:
- ^MMTH = % stocks above 200 DMA (direct Yahoo symbol!)
- C:ISSU = NYSE advance/decline data
- CNN endpoint accepts date param for historical data
- CBOE CSVs have data back to 2003
- 33 total calls per seed run, ~6s runtime

All 10 categories now have confirmed free sources.

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* Rewrite F&G brief as forward-looking design doc

Remove all reverse-engineering language, screenshot references, and
discovery notes. Clean structure: goal, scoring model, data sources,
formulas, seed script plan, implementation phases, MVP path.

https://claude.ai/code/session_01HR69u6oF1VCMwsC2PHFL8i

* docs: apply gold standard corrections to fear-greed-index-2.0 brief

* feat(market): add Fear & Greed Index 2.0 — 10-category composite sentiment panel

Composite 0-100 index from 10 weighted categories: sentiment (CNN F&G,
AAII, crypto F&G), volatility (VIX, term structure), positioning (P/C
ratio, SKEW), trend (SPX vs MAs), breadth (% >200d, RSP/SPY divergence),
momentum (sector RSI, ROC), liquidity (M2, Fed BS, SOFR), credit (HY/IG
spreads), macro (Fed rate, yield curve, unemployment), cross-asset
(gold/bonds/DXY vs equities).

Data layer:
- seed-fear-greed.mjs: 19 Yahoo symbols (150ms gaps), CBOE P/C CSVs,
  CNN F&G API, AAII scrape (degraded-safe), FRED Redis reads. TTL 64800s.
- seed-economy.mjs: add BAMLC0A0CM (IG spread) and SOFR to FRED_SERIES.
- Bootstrap 4-file checklist: cache-keys, bootstrap.js, health.js, handler.

Proto + RPC:
- get_fear_greed_index.proto with FearGreedCategory message.
- get-fear-greed-index.ts handler reads seeded Redis data.

Frontend:
- FearGreedPanel with gauge, 9-metric header grid, 10-category breakdown.
- Self-loading via bootstrap hydration + RPC fallback.
- Registered in panel-layout, App.ts (prime + refresh), panel config,
  Cmd-K commands, finance variant, i18n (en/ar/zh/es).

* fix(market): add RPC_CACHE_TIER entry for get-fear-greed-index

* fix(docs): escape bare angle bracket in fear-greed brief for MDX

* fix(docs): fix markdown lint errors in fear-greed brief (blank lines around headings/lists)

* fix(market): fix seed-fear-greed bugs from code review

- fredLatest/fredNMonthsAgo: guard parseFloat with Number.isFinite to
  handle FRED's "." missing-data sentinel (was returning NaN which
  propagated through scoring as a truthy non-null value)
- Remove 3 unused Yahoo symbols (^NYA, HYG, LQD) that were fetched
  but not referenced in any scoring category (saves ~450ms per run)
- fedRateStr: display effective rate directly instead of deriving
  target range via (fedRate - 0.25) which was incorrect

* fix(market): address P2/P3 review findings in Fear & Greed

- FearGreedPanel: add mapSeedPayload() to correctly map raw seed
  JSON to proto-shaped FearGreedData; bootstrap hydration was always
  falling through to RPC because seed shape (composite.score) differs
  from proto shape (compositeScore)
- FearGreedPanel: fix fmt() — remove === 0 guard and add explicit
  > 0 checks on VIX and P/C Ratio display to handle proto default
  zeros without masking genuine zero values (e.g. pctAbove200d)
- seed-fear-greed: remove broken history write — each run overwrote
  the key with a single-entry array (no read-then-append), making the
  90-day TTL meaningless; no consumer exists yet so defer to later
- seed-fear-greed: extract hySpreadVal const to avoid double fredLatest call
- seed-fear-greed: fix stale comment (19 symbols → 16 after prior cleanup)

---------

Co-authored-by: Claude <noreply@anthropic.com>
2026-03-24 09:45:59 +04:00

16 KiB
Raw Permalink Blame History

Fear & Greed Index 2.0 — Design Brief

Goal

Build a composite market sentiment gauge (0100) combining 10 weighted categories into a single score. Unlike CNN's Fear & Greed Index (~7 inputs, widely criticized for lagging and oversimplifying), this uses 10 granular categories with more inputs per category to produce a nuanced, institutional-quality reading.


Composite Score

Final Score = Σ (Category_Score × Category_Weight)

Each category scores 0100 (0 = Extreme Fear, 100 = Extreme Greed). The weighted sum produces the composite index.

10 Categories

# Category Weight What It Measures
1 Sentiment 10% CNN F&G, AAII Bull/Bear surveys, crypto sentiment
2 Volatility 10% VIX level, VIX term structure (contango/backwardation)
3 Positioning 15% Put/Call ratios, options skew (CBOE SKEW)
4 Trend 10% SPX vs 20d/50d/200d MAs, price momentum
5 Breadth 10% % stocks > 200 DMA, advance/decline ratio, equal-weight divergence
6 Momentum 10% Sector RSI spread, rate of change
7 Liquidity 15% M2 growth, Fed balance sheet, SOFR rate
8 Credit 10% HY spreads, IG spreads, credit ETF trends
9 Macro 5% Fed rate, yield curve, unemployment
10 Cross-Asset 5% Gold/USD correlation, bonds vs equities

Score Labels

Range Label
020 Extreme Fear
2040 Fear
4060 Neutral
6080 Greed
80100 Extreme Greed

Header Metrics (9 key stats)

Metric Source Context
CNN F&G CNN dataviz API 0100 score + label
AAII Bear % AAII survey vs historical average
AAII Bull % AAII survey vs historical average
Put/Call Ratio CBOE CDN CSV vs 1yr average
VIX Yahoo / FRED % change
HY Spread FRED vs long-term average
% > 200 DMA Yahoo ^MMTH vs recent peak
10Y Yield FRED level
Fed Rate FRED current range

Data Sources

All sources are free with no paid API keys required.

Already Available (read from Redis)

Data Point FRED Series Used In
VIX VIXCLS Volatility
HY Spread (OAS) BAMLH0A0HYM2 Credit
10Y Yield DGS10 Macro
Fed Funds Rate FEDFUNDS Macro
10Y-2Y Spread T10Y2Y Macro
M2 Money Supply M2SL Liquidity
Fed Balance Sheet WALCL Liquidity
Unemployment UNRATE Macro
Crypto Fear & Greed Alternative.me (macro-signals) Sentiment

New FRED Series (add to seed-economy.mjs)

Series Name Category
BAMLC0A0CM ICE BofA US IG OAS Credit
SOFR Secured Overnight Financing Rate Liquidity

New External Sources

Source Endpoint Format Auth Reliability
CNN Fear & Greed production.dataviz.cnn.io/index/fearandgreed/graphdata/{date} JSON User-Agent header MEDIUM
AAII Sentiment aaii.com/sentimentsurvey (HTML scrape) HTML User-Agent header LOW (blocks bots)
CBOE Total P/C cdn.cboe.com/.../totalpc.csv CSV None HIGH
CBOE Equity P/C cdn.cboe.com/.../equitypc.csv CSV None HIGH

Yahoo Finance Symbols (19 total)

Uses query1.finance.yahoo.com/v8/finance/chart — no API key, User-Agent header only.

# Symbol Category Purpose
1 ^GSPC Trend, Momentum SPX — compute 20/50/200 DMA, ROC
2 ^VIX Volatility Real-time VIX
3 ^VIX9D Volatility 9-day VIX for term structure
4 ^VIX3M Volatility 3-month VIX for term structure
5 ^SKEW Positioning CBOE SKEW index
6 ^MMTH Breadth % of stocks above 200 DMA
7 ^NYA Breadth NYSE Composite for breadth divergence
8 C:ISSU Breadth NYSE advances/declines/unchanged
9 GLD Cross-Asset Gold proxy
10 TLT Cross-Asset Bonds proxy
11 SPY Cross-Asset, Breadth Equity benchmark
12 RSP Breadth Equal-weight S&P 500 (vs SPY divergence)
13 DX-Y.NYB Cross-Asset USD Dollar Index
14 HYG Credit HY bond ETF trend
15 LQD Credit IG bond ETF trend
16 XLK Momentum Tech sector
17 XLF Momentum Financial sector
18 XLE Momentum Energy sector
19 XLV Momentum Healthcare sector

Notes:

  • ^MMTH = % above 200-day MA (not ^MMTW which is 20-day)
  • C:ISSU = NYSE advance/decline/unchanged data. Unvalidated via /v8/finance/chart endpoint — must confirm it returns advance/decline figures before relying on it. If unavailable, Breadth drops ad_score and reweights: breadth_score * 0.57 + rsp_score * 0.43
  • Fallback: Finnhub candle API for ETF symbols; breadth symbols Yahoo-only

Scoring Formulas

1. Sentiment (10%)

inputs: CNN_FG, AAII_Bull, AAII_Bear  (AAII is LOW reliability — blocks bots)

// Normal path (AAII available):
score = (CNN_FG * 0.4) + (AAII_Bull_Percentile * 0.3) + ((100 - AAII_Bear_Percentile) * 0.3)

// Degraded path (AAII unavailable — store aaiBull/aaiBear as null, not 0):
score = CNN_FG  // 100% weight on CNN F&G; crypto F&G from Redis as secondary signal if CNN also fails
// aaiBull and aaiBear fields: null (not 0 — zero skews score toward Extreme Fear)

Reliability notes: CNN F&G is MEDIUM reliability. If both CNN and AAII fail, use cryptoFearGreed from Redis (already seeded via macro-signals) as a proxy — it is directionally correlated. Mark unavailable: true only if all three sentiment sources are absent.

2. Volatility (10%)

inputs: VIX, VIX_Term_Structure
vix_score = clamp(100 - ((VIX - 12) / 28) * 100, 0, 100)  // VIX 12=100, VIX 40=0
term_score = contango ? 70 : backwardation ? 30 : 50
score = vix_score * 0.7 + term_score * 0.3

3. Positioning (15%)

inputs: Put_Call_Ratio, Options_Skew
pc_score = clamp(100 - ((PC_Ratio - 0.7) / 0.6) * 100, 0, 100)  // 0.7=greed, 1.3=fear
skew_score = clamp(100 - ((SKEW - 100) / 50) * 100, 0, 100)
score = pc_score * 0.6 + skew_score * 0.4

4. Trend (10%)

inputs: SPX_Price, SMA20, SMA50, SMA200
above_count = count(price > SMA20, price > SMA50, price > SMA200)
distance_200 = (price - SMA200) / SMA200
score = (above_count / 3) * 50 + clamp(distance_200 * 500 + 50, 0, 100) * 0.5

5. Breadth (10%)

inputs: Pct_Above_200DMA, Advance_Decline, RSP_SPY_Divergence
breadth_score = Pct_Above_200DMA  // already 0-100
ad_score = clamp((AD_Ratio - 0.5) / 1.5 * 100, 0, 100)
rsp_score = clamp(RSP_SPY_30d_diff * 10 + 50, 0, 100)
score = breadth_score * 0.4 + ad_score * 0.3 + rsp_score * 0.3

6. Momentum (10%)

inputs: Sector_RSI_Spread, SPX_ROC_20d
rsi_score = clamp((avg_sector_rsi - 30) / 40 * 100, 0, 100)
roc_score = clamp(SPX_ROC_20d * 10 + 50, 0, 100)
score = rsi_score * 0.5 + roc_score * 0.5

7. Liquidity (15%)

inputs: M2_YoY_Change, Fed_Balance_Sheet_Change, SOFR_Rate
m2_score = clamp(M2_YoY * 10 + 50, 0, 100)
fed_score = clamp(Fed_BS_MoM * 20 + 50, 0, 100)
sofr_score = clamp(100 - SOFR * 15, 0, 100)
score = m2_score * 0.4 + fed_score * 0.3 + sofr_score * 0.3

8. Credit (10%)

inputs: HY_Spread, IG_Spread, HY_Spread_Change_30d
hy_score = clamp(100 - ((HY_Spread - 3.0) / 5.0) * 100, 0, 100)
ig_score = clamp(100 - ((IG_Spread - 0.8) / 2.0) * 100, 0, 100)
trend_score = HY_narrowing ? 70 : HY_widening ? 30 : 50
score = hy_score * 0.4 + ig_score * 0.3 + trend_score * 0.3

9. Macro (5%)

inputs: Fed_Rate, Yield_Curve_10Y2Y, Unemployment_Trend
rate_score = clamp(100 - Fed_Rate * 15, 0, 100)
curve_score = T10Y2Y > 0 ? 60 + T10Y2Y * 20 : 40 + T10Y2Y * 40
unemp_score = clamp(100 - (UNRATE - 3.5) * 20, 0, 100)
score = rate_score * 0.3 + curve_score * 0.4 + unemp_score * 0.3

10. Cross-Asset (5%)

inputs: Gold_vs_SPY_30d, TLT_vs_SPY_30d, DXY_30d_Change
gold_signal = Gold_30d > SPY_30d ? fear : greed
bond_signal = TLT_30d > SPY_30d ? fear : greed
dxy_signal = DXY_rising ? slight_fear : slight_greed
score = weighted combination with mean reversion

Computed Metrics (derived from fetched data, no extra API calls)

Metric Inputs Formula Category
SPX 20/50/200 DMA ^GSPC closes smaCalc(prices, period) Trend
SPX ROC 20d ^GSPC closes rateOfChange(prices, 20) Momentum
VIX Term Structure ^VIX, ^VIX9D, ^VIX3M VIX/VIX3M ratio (<1 = contango) Volatility
Sector RSI (14d) XLK/XLF/XLE/XLV Standard RSI formula Momentum
Cross-asset 30d returns GLD, TLT, SPY, DXY rateOfChange(prices, 30) Cross-Asset
M2 YoY change M2SL (latest - 12mo_ago) / 12mo_ago Liquidity
Fed BS MoM change WALCL (latest - 4wk_ago) / 4wk_ago Liquidity
HY spread trend BAMLH0A0HYM2 30d change direction Credit
RSP/SPY ratio RSP, SPY RSP_return_30d - SPY_return_30d Breadth

Seed Script: seed-fear-greed.mjs

Follows the existing pattern: Railway cron → fetch external APIs → compute scores → atomic publish to Redis → server handler reads from Redis.

Redis Keys

market:fear-greed:v1              # Composite index + all category scores
market:fear-greed:history:v1      # Sorted set — daily snapshots for sparklines (score UNIX score, member ISO date string)
seed-meta:market:fear-greed       # Metadata (fetchedAt, recordCount, sourceVersion)
seed-lock:market:fear-greed       # Concurrency lock

TTL: 64800s (18h) — 3× the 6h cron interval. Required to survive 2 missed cron cycles (Railway downtime, deploy gaps). runSeed() extends this same TTL on both fetch-failure and empty-data paths. Cron: 0 0,6,12,18 * * * (every 6h) health.js maxStaleMin: 720 (12h) — 2× interval. One missed cycle never fires a spurious WARN; the 20min self-heal from runSeed() retry covers transient failures.

composite.previous requires a pre-write Redis GET. Before calling runSeed(), read market:fear-greed:v1 from Redis, extract composite.score, pass it into publishTransform as previous. runSeed() then overwrites the key atomically. Do NOT compute previous after the write — the key is already overwritten.

API Call Budget

Source Calls Rate Limited? Auth
Yahoo Finance 19 symbols 150ms gaps User-Agent only
CBOE CDN 2 CSVs No None
CNN dataviz 1 No User-Agent only
AAII 1 Blocks bots User-Agent + scrape
Redis reads ~10 FRED series No Bearer token
Total ~33

Estimated runtime: ~3s (Yahoo sequential) + ~2s (CBOE/CNN/AAII parallel) + ~1s (Redis) = ~6s per run

Timeouts: Set AbortSignal.timeout(8000) on AAII scrape (frequently stalls). AAII failure must not block the entire seed run — wrap in try/catch, log warn, continue with degraded Sentiment scoring.

Output Schema (stored in Redis)

{
  "timestamp": "2026-03-24T12:00:00Z",
  "composite": {
    "score": 38.7,
    "label": "Fear",
    "previous": 41.2
  },
  "categories": {
    "sentiment": { "score": 19, "weight": 0.10, "contribution": 1.9, "inputs": { "cnnFearGreed": 16, "cnnLabel": "Extreme Fear", "aaiBull": 30.4, "aaiBear": 52.0 }, "degraded": false },
    // degraded: true when AAII unavailable; aaiBull/aaiBear: null (not 0) when AAII fetch fails
    "volatility": { "score": 47, "weight": 0.10, "contribution": 4.7, "inputs": { "vix": 26.78, "vixChange": 11.31, "vix9d": 28.1, "vix3m": 24.5, "termStructure": "backwardation" } },
    "positioning": { "score": 34, "weight": 0.15, "contribution": 5.1, "inputs": { "putCallRatio": 1.01, "putCallAvg": 0.87, "skew": 135 } },
    "trend": { "score": 52, "weight": 0.10, "contribution": 5.2, "inputs": { "spxPrice": 5667, "sma20": 5580, "sma50": 5520, "sma200": 5200, "aboveMaCount": 3 } },
    "breadth": { "score": 40, "weight": 0.10, "contribution": 4.0, "inputs": { "pctAbove200d": 43.93, "rspSpyRatio": -2.1, "advDecRatio": 0.85 } },
    "momentum": { "score": 13, "weight": 0.10, "contribution": 1.3, "inputs": { "spxRoc20d": -3.2, "sectorRsiAvg": 38, "leadersVsLaggards": -12.5 } },
    "liquidity": { "score": 26, "weight": 0.15, "contribution": 3.9, "inputs": { "m2Yoy": 1.2, "fedBsMom": -0.8, "sofr": 5.31 } },
    "credit": { "score": 68, "weight": 0.10, "contribution": 6.8, "inputs": { "hySpread": 3.27, "igSpread": 1.15, "hyTrend30d": "narrowing" } },
    "macro": { "score": 44, "weight": 0.05, "contribution": 2.2, "inputs": { "fedRate": 3.625, "t10y2y": 0.15, "unrate": 4.1 } },
    "crossAsset": { "score": 72, "weight": 0.05, "contribution": 3.6, "inputs": { "goldReturn30d": 4.2, "tltReturn30d": 1.8, "spyReturn30d": -2.1, "dxyChange30d": -1.5 } }
  },
  "headerMetrics": {
    "cnnFearGreed": { "value": 16, "label": "Extreme Fear" },
    "aaiBear": { "value": 52, "context": "6-wk high" },
    "aaiBull": { "value": 30.4, "context": "Below avg" },
    "putCall": { "value": 1.01, "context": "vs 0.87 yr avg" },
    "vix": { "value": 26.78, "context": "+11.31%" },
    "hySpread": { "value": 3.27, "context": "vs LT avg" },
    "pctAbove200d": { "value": 43.93, "context": "Down from 68.5%" },
    "yield10y": { "value": 4.25 },
    "fedRate": { "value": "3.50-3.75%" }
  },
  "unavailable": false
}

Implementation Plan

Phase 1: Data Layer

  1. Add BAMLC0A0CM and SOFR to seed-economy.mjs FRED_SERIES array
    • Note: SOFR is weekly cadence from FRED, not daily — Liquidity formula is stable between releases
  2. Validate C:ISSU symbol returns advance/decline data via Yahoo /v8/finance/chart — confirm before building Breadth formula around it
  3. Create seed-fear-greed.mjs:
    • TTL: 64800s (18h = 3× interval)
    • AAII fetch: AbortSignal.timeout(8000), wrapped in try/catch — failure uses degraded Sentiment scoring
    • Pre-write step: GET market:fear-greed:v1 from Redis, extract composite.score as previous, pass via publishTransform
    • runSeed() calls process.exit(0) — all extra key writes (e.g. history key) must use the extraKeys option, NOT code after the runSeed() call
  4. Register with bootstrap 4-file checklist:
    • cache-keys.ts — add market:fear-greed:v1
    • api/bootstrap.js — register the key
    • health.js — classify as BOOTSTRAP_KEYS (seeded, CRIT if empty); set maxStaleMin: 720 (12h = 2× interval)
    • gateway.ts — wire GetFearGreedIndex RPC

Phase 2: Proto + RPC

  1. New proto: proto/worldmonitor/market/v1/fear_greed.proto
    • GetFearGreedIndex RPC
    • Messages for composite score, category scores, and header metrics
  2. New handler: server/worldmonitor/market/v1/get-fear-greed-index.ts
    • Reads computed data from Redis, returns structured response

Phase 3: Frontend Panel

  1. New component: src/components/FearGreedPanel.ts
    • Gauge — semicircular 0100 dial with color gradient (red→yellow→green)
    • Header grid — 9 key metrics with contextual annotations
    • Category breakdown — expandable cards per category (score, weight, contribution, bar)
    • Handle degraded: true on Sentiment card (show "AAII unavailable" note)
  2. Register in finance variant panel config

Phase 4: Polish

  1. Historical sparklines — append daily snapshot to market:fear-greed:history:v1 (sorted set, score = UNIX timestamp, member = ISO date + composite score JSON). Write via extraKeys in Phase 1 seeder. TTL: 90 days (7776000s). Frontend reads this key for trend sparkline.
  2. Alerts on threshold crossings (e.g. score drops below 20)

MVP Path

Build the initial version using only data we already have + easy additions:

  1. Volatility — VIX from FRED
  2. Credit — HY + IG spread from FRED
  3. Macro — Fed rate + yield curve + unemployment from FRED
  4. Trend — SPX price vs computed MAs from Yahoo
  5. Liquidity — M2 + Fed balance sheet from FRED + SOFR
  6. Sentiment — CNN F&G endpoint + crypto F&G (already have)
  7. Momentum — Sector ETF returns from Yahoo
  8. Cross-Asset — GLD/TLT/SPY/DXY returns from Yahoo
  9. Positioning — CBOE put/call CSVs + SKEW from Yahoo
  10. Breadth — ^MMTH + RSP/SPY divergence + C:ISSU from Yahoo

All 10 categories covered from day one. No paid sources needed.