worldmonitor/todos/054-pending-p2-reddit-permalink-no-url-scheme-validation.md at e7ba05553df5d412d6444b9c955c44907112e521

mirror of https://github.com/koala73/worldmonitor.git synced 2026-04-25 17:14:57 +02:00

Files

Elie Habib e7ba05553d fix(health): disease outbreaks CDC/Outbreak feeds, VPD tracker seed, BOOTSTRAP_KEYS gold standard (#2378 )

* feat(panels): Disease Outbreaks, Shipping Stress, Social Velocity, nuclear test site monitoring

- Add HealthService proto with ListDiseaseOutbreaks RPC (WHO + ProMED RSS)
- Add GetShippingStress RPC to SupplyChainService (Yahoo Finance carrier ETFs)
- Add GetSocialVelocity RPC to IntelligenceService (Reddit r/worldnews + r/geopolitics)
- Enrich earthquake seed with Haversine nuclear test-site proximity scoring
- Add 5 nuclear test sites to NUCLEAR_FACILITIES (Punggye-ri, Lop Nur, Novaya Zemlya, Nevada NTS, Semipalatinsk)
- Add shipping stress + social velocity seed loops to ais-relay.cjs
- Add seed-disease-outbreaks.mjs Railway cron script
- Wire all new RPCs: edge functions, handlers, gateway cache tiers, health.js STANDALONE_KEYS/SEED_META

* fix(relay): apply gold standard retry/TTL-extend pattern to shipping-stress and social-velocity seeders

* fix(review): address all PR #2375 review findings

- health.js: shippingStress maxStaleMin 30→45 (3x interval), socialVelocity 20→30 (3x interval)
- health.js: remove shippingStress/diseaseOutbreaks/socialVelocity from ON_DEMAND_KEYS (relay/cron seeds, not on-demand)
- cache-keys.ts: add shippingStress, diseaseOutbreaks, socialVelocity to BOOTSTRAP_CACHE_KEYS
- ais-relay.cjs: stressScore formula 50→40 (neutral market = moderate, not elevated)
- ais-relay.cjs: fetchedAt Date.now() (consistent with other seeders)
- ais-relay.cjs: deduplicate cross-subreddit article URLs in social velocity loop
- seed-disease-outbreaks.mjs: WHO URL → specific DON RSS endpoint (not dead general news feed)
- seed-disease-outbreaks.mjs: validate() requires outbreaks.length >= 1 (reject empty array)
- seed-disease-outbreaks.mjs: stable id using hash(link) not array index
- seed-disease-outbreaks.mjs: RSS regexes use [\s\S]*? for CDATA multiline content
- seed-earthquakes.mjs: Lop Nur coordinates corrected (41.39,89.03 not 41.75,88.35)
- seed-earthquakes.mjs: sourceVersion bumped to usgs-4.5-day-nuclear-v1
- earthquake.proto: fields 8-11 marked optional (distinguish not-enriched from enriched=false/0)
- buf generate: regenerate seismology service stubs

* revert(cache-keys): don't add new keys to bootstrap without frontend consumers

* fix(panels): address all P1/P2/P3 review findings for PR #2375

- proto: add INT64_ENCODING_NUMBER annotation + sebuf import to get_shipping_stress.proto (run make generate)
- bootstrap: register shippingStress (fast), socialVelocity (fast), diseaseOutbreaks (slow) in api/bootstrap.js + cache-keys.ts
- relay: update WIDGET_SYSTEM_PROMPT with new bootstrap keys and live RPCs for health/supply-chain/intelligence
- seeder: remove broken ProMED feed URL (promedmail.org/feed/ returns HTML 404); add 500K size guard to fetchRssItems; replace private COUNTRY_CODE_MAP with shared geo-extract.mjs; remove permanently-empty location field; bump sourceVersion to who-don-rss-v2
- handlers: remove dead .catch from all 3 new RPC handlers; fix stressLevel fallback to low; fix fetchedAt fallback to 0
- services: add fetchShippingStress, disease-outbreaks.ts, social-velocity.ts with getHydratedData consumers

* fix(health): move seeded keys to BOOTSTRAP_KEYS, add VPD tracker seed and feeds

- Reclassify diseaseOutbreaks, shippingStress, socialVelocity from
  STANDALONE_KEYS to BOOTSTRAP_KEYS so health endpoint reports CRIT
  (not WARN) when their seeds miss a cycle
- Add vpdTrackerRealtime and vpdTrackerHistorical to BOOTSTRAP_KEYS
  with SEED_META entries (maxStaleMin: 2880 = 2x daily interval)
- Fix seed-disease-outbreaks: add CDC and Outbreak News Today feeds
  alongside WHO, populate location field from title parsing, fix TTL
  to 259200s (3x daily interval per gold standard)
- Add seed-vpd-tracker.mjs: scrapes Think Global Health VPD Tracker
  bundle (1,827 realtime alerts + 25,960 historical WHO records),
  writes both Redis keys in one runSeed call via extraKeys
- Add review todos 049-059 from PR #2375 code review

2026-03-27 22:47:24 +04:00

2.6 KiB

Raw Blame History

status, priority, issue_id, tags, dependencies

status

priority

issue_id

Problem Statement

The seedSocialVelocity loop in scripts/ais-relay.cjs stores p.permalink from Reddit API responses directly into Redis without validating the URL scheme. Reddit permalinks are typically relative paths (e.g., /r/worldnews/comments/...) but the code prepends https://reddit.com — however, if the Reddit API ever returns a full URL with a different scheme (e.g., javascript: or data:), that value would be stored and potentially rendered as a link in the Social Velocity panel, creating an XSS vector.

Findings

File: scripts/ais-relay.cjs — seedSocialVelocity section: url: 'https://reddit.com' + p.permalink (or similar construction)
Concern: p.permalink from the Reddit JSON API is typically a relative path starting with /r/, but this is not validated
Impact (if exploited): If a future Reddit API change or edge case returns a full URL in permalink, the stored value could contain an arbitrary scheme. Frontend rendering the URL without validation could execute JavaScript
Secondary concern: p.permalink from upvote-manipulated posts could contain unicode path segments that normalize unexpectedly

Proposed Solutions

Option A: Validate permalink starts with /r/ before storing (Recommended)

const safePermalink = p.permalink?.startsWith('/r/') ? p.permalink : null;
if (!safePermalink) continue; // skip malformed items
const url = 'https://reddit.com' + safePermalink;

Effort: Trivial (one guard)
Risk: None — drops malformed items, logs warning

Option B: Parse full URL and assert scheme is https

const url = 'https://reddit.com' + p.permalink;
try {
  const parsed = new URL(url);
  if (parsed.protocol !== 'https:') continue;
} catch { continue; }

Effort: Trivial
Risk: None

Option C: Sanitize on the frontend rendering side

Ensure the Social Velocity panel only renders URLs with https: scheme. Belt-and-suspenders approach alongside server-side validation.

Effort: Small
Risk: None — defense in depth

Acceptance Criteria

p.permalink validated (must start with /r/ or parsed URL must have https: scheme) before storage
Items with invalid permalinks are skipped with a console.warn
Frontend Social Velocity panel does not render non-https URLs as clickable links

Work Log

2026-03-27: Identified by security-sentinel agent during PR #2375 review.

2.6 KiB Raw Blame History