mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
chore: bump version to 2.5.22 & comprehensive README update (#706)
Bump version 2.5.21 → 2.5.22 across package.json, Cargo.toml, and tauri.conf.json. README: document 15+ recently shipped features that were missing from the README — AI Deduction panel, Headline Memory (RAG), server-side feed aggregation, Gulf Economies panel, TV Mode, mobile map with touch gestures, fullscreen live video, 18+ HLS channels, breaking news click-through, badge animation toggle, cache purge admin endpoint, locale-aware feed boost, OREF Redis persistence + 1,478 Hebrew→English translations, and Oceania region tab. Update PostHog → Vercel Analytics. Add 21 new completed roadmap items.
This commit is contained in:
Elie Habib
138
README.md
138
README.md
@@ -40,8 +40,8 @@
|
||||
| Information overload | **AI-synthesized briefs** with focal point detection and local LLM support |
|
||||
| Crypto/macro signal noise | **7-signal market radar** with composite BUY/CASH verdict |
|
||||
| Expensive OSINT tools ($$$) | **100% free & open source** |
|
||||
| Static news feeds | **Real-time updates** with live video streams |
|
||||
| Cloud-dependent AI tools | **Run AI locally** with Ollama/LM Studio — no API keys, no data leaves your machine |
|
||||
| Static news feeds | **Real-time updates** with live video streams and AI-powered deductions |
|
||||
| Cloud-dependent AI tools | **Run AI locally** with Ollama/LM Studio — no API keys, no data leaves your machine. Opt-in **Headline Memory** builds a local semantic index of every headline for RAG-powered queries |
|
||||
| Web-only dashboards | **Native desktop app** (Tauri) for macOS, Windows, and Linux + installable PWA with offline map support |
|
||||
| Flat 2D maps | **3D WebGL globe** with deck.gl rendering and 40+ toggleable data layers |
|
||||
| Siloed financial data | **Finance variant** with 92 stock exchanges, 19 financial centers, 13 central banks, BIS data, WTO trade policy, and Gulf FDI tracking |
|
||||
@@ -68,7 +68,7 @@ All four variants run from a single codebase — switch between them with one cl
|
||||
|
||||
- **Multilingual UI** — Fully localized interface supporting **16 languages: English, French, Spanish, German, Italian, Polish, Portuguese, Dutch, Swedish, Russian, Arabic, Chinese, Japanese, Turkish, Thai, and Vietnamese**. Language bundles are lazy-loaded on demand — only the active language is fetched, keeping initial bundle size minimal.
|
||||
- **RTL Support** — Native right-to-left layout support for Arabic (`ar`) and Hebrew.
|
||||
- **Localized News Feeds** — Region-specific RSS selection based on language preference (e.g., viewing the app in French loads Le Monde, Jeune Afrique, and France24). Seven locales have dedicated native-language feed sets: French, Arabic, German, Spanish, Turkish (BBC Türkçe, Hurriyet, DW Turkish), Polish (TVN24, Polsat News, Rzeczpospolita), Russian (BBC Russian, Meduza, Novaya Gazeta Europe), Thai (Bangkok Post, Thai PBS), and Vietnamese (VnExpress, Tuoi Tre News).
|
||||
- **Localized News Feeds** — Region-specific RSS selection based on language preference (e.g., viewing the app in French loads Le Monde, Jeune Afrique, and France24). 17 locales have dedicated native-language feed sets: French, Arabic, German, Spanish, Italian, Dutch, Swedish, Turkish (BBC Türkçe, Hurriyet, DW Turkish), Polish (TVN24, Polsat News, Rzeczpospolita), Russian (BBC Russian, Meduza, Novaya Gazeta Europe), Thai (Bangkok Post, Thai PBS), Vietnamese (VnExpress, Tuoi Tre News), Korean (Yonhap, Chosun Ilbo), Greek (Kathimerini, Naftemporiki, in.gr, Proto Thema), Portuguese (O Globo, Folha), Japanese (Asahi Shimbun), and Chinese (MIIT, MOFCOM). On first load for non-English users, a one-time locale boost automatically enables these native-language sources without overwriting manual preferences.
|
||||
- **AI Translation** — Integrated LLM translation for news headlines and summaries, enabling cross-language intelligence gathering.
|
||||
- **Regional Intelligence** — Dedicated monitoring panels for Africa, Latin America, Middle East, and Asia with local sources.
|
||||
|
||||
@@ -82,11 +82,15 @@ All four variants run from a single codebase — switch between them with one cl
|
||||
- **8 regional presets** — Global, Americas, Europe, MENA, Asia, Africa, Oceania, Latin America
|
||||
- **Time filtering** — 1h, 6h, 24h, 48h, 7d event windows
|
||||
- **URL state sharing** — map center, zoom, active layers, and time range are encoded in the URL for shareable views (`?view=mena&zoom=4&layers=conflicts,bases`)
|
||||
- **Mobile touch gestures** — single-finger pan with inertial velocity animation (0.92 decay factor, computed from 4-entry circular touch history), two-finger pinch-to-zoom with center-point preservation, and bottom-sheet popups with drag-to-dismiss. An 8px movement threshold prevents accidental interaction during taps
|
||||
- **Timezone-based region detection** — on first load, the map centers on the user's approximate region derived from `Intl.DateTimeFormat().resolvedOptions().timeZone` — no network dependency, no geolocation prompt. If the Geolocation permission is already granted, it upgrades to precise coordinates silently
|
||||
|
||||
### AI-Powered Intelligence
|
||||
|
||||
- **World Brief** — LLM-synthesized summary of top global developments with a 4-tier provider fallback chain: Ollama (local) → Groq (cloud) → OpenRouter (cloud) → browser-side T5 (Transformers.js). Each tier is attempted with a 5-second timeout before falling through to the next, so the UI is never blocked. Results are Redis-cached (24h TTL) and content-deduplicated so identical headlines across concurrent users trigger exactly one LLM call
|
||||
- **Local LLM Support** — Ollama and LM Studio (any OpenAI-compatible endpoint) run AI summarization entirely on local hardware. No API keys required, no data leaves the machine. The desktop app auto-discovers available models from the local instance and populates a selection dropdown, filtering out embedding-only models. Default fallback model: `llama3.1:8b`
|
||||
- **AI Deduction & Forecasting** — an interactive geopolitical analysis tool where analysts enter a free-text query (e.g., "What will happen in the next 24 hours in the Middle East?") and receive an LLM-generated near-term timeline deduction. The panel auto-populates context from the 15 most recent live headlines via `buildNewsContext()`, so the AI always has current situational awareness. Other panels can pre-fill and auto-submit queries via the `wm:deduct-context` custom event for seamless cross-panel deep-linking into contextual forecasts. Results are Redis-cached (1-hour TTL) by query hash to avoid redundant LLM calls
|
||||
- **Headline Memory (RAG)** — an opt-in client-side Retrieval-Augmented Generation system. When enabled in Settings, every incoming RSS headline is embedded using an ONNX model (`all-MiniLM-L6-v2`, 384-dimensional float32 vectors) running in a dedicated Web Worker, then stored in IndexedDB (`worldmonitor_vector_store`, capped at 5,000 vectors with LRU eviction by ingestion time). Any component can semantically search the headline archive using natural-language queries — results are ranked by brute-force cosine similarity and returned in score order. The entire pipeline runs locally in the browser with zero server dependency, enabling persistent semantic intelligence across sessions
|
||||
- **Hybrid Threat Classification** — instant keyword classifier with async LLM override for higher-confidence results
|
||||
- **Focal Point Detection** — correlates entities across news, military activity, protests, outages, and markets to identify convergence
|
||||
- **Country Instability Index** — real-time stability scores for every country with incoming data using weighted multi-signal blend. 23 curated tier-1 nations have tuned baseline risk profiles; all other countries receive universal scoring with sensible defaults when any event data (protests, conflicts, outages, displacement, climate anomalies) is detected
|
||||
@@ -141,6 +145,7 @@ All four variants run from a single codebase — switch between them with one cl
|
||||
<summary><strong>Market & Crypto Intelligence</strong></summary>
|
||||
|
||||
- 7-signal macro radar with composite BUY/CASH verdict
|
||||
- **Gulf Economies panel** — live data for GCC financial markets across three sections: **Indices** (Tadawul/Saudi Arabia, Dubai Financial Market, Abu Dhabi, Qatar, WisdomTree Gulf Dividend, Muscat MSM 30), **Currencies** (SAR, AED, QAR, KWD, BHD, OMR vs USD), and **Oil** (WTI Crude, Brent Crude). All quotes fetched from Yahoo Finance with staggered batching, Redis-cached for 8 minutes, with mini sparklines per quote and 60-second polling
|
||||
- Real-time crypto prices (BTC, ETH, SOL, XRP, and more) via CoinGecko
|
||||
- BTC spot ETF flow tracker (IBIT, FBTC, GBTC, and 7 more)
|
||||
- Stablecoin peg health monitor (USDT, USDC, DAI, FDUSD, USDe)
|
||||
@@ -177,14 +182,17 @@ All four variants run from a single codebase — switch between them with one cl
|
||||
|
||||
### Live News & Video
|
||||
|
||||
- **150+ RSS feeds** across geopolitics, defense, energy, tech, and finance — domain-allowlisted proxy prevents CORS issues. Each variant loads its own curated feed set: ~25 categories for geopolitical, ~20 for tech, ~18 for finance
|
||||
- **8+ default live video streams** — Bloomberg, Sky News, Al Jazeera, Euronews, DW, France24, CNBC, Al Arabiya — with automatic live detection that scrapes YouTube channel pages every 5 minutes to find active streams. 30+ additional channels available from an expandable library (Fox, BBC, CNN Turk, TRT, RT, CBS, NBC, CNN Brasil, and more)
|
||||
- **HLS native streaming** — 10 channels (Sky News, Euronews, DW, France24, Al Arabiya, CBS News, TRT World, Sky News Arabia, Al Hadath, RT) stream via native HLS `<video>` elements instead of YouTube iframes, bypassing cookie popups, bot checks, and WKWebView autoplay restrictions. HLS failure triggers automatic 5-minute cooldown with YouTube iframe fallback. RT (Russia Today) — banned from YouTube — streams exclusively via HLS
|
||||
- **150+ RSS feeds** across geopolitics, defense, energy, tech, and finance — domain-allowlisted proxy prevents CORS issues. Each variant loads its own curated feed set: ~25 categories for geopolitical, ~20 for tech, ~18 for finance. **Server-side feed aggregation** — a single `listFeedDigest` RPC call fetches all feeds server-side (batched at 20 concurrent requests with 8-second per-feed timeouts and a 25-second overall deadline), caches the categorized digest in Redis for 15 minutes, and serves it to all clients. This eliminates per-client feed fan-out, reducing Vercel Edge invocations by approximately 95%. Individual feed results are separately cached for 10 minutes, so repeated digest builds within that window reuse previously fetched content
|
||||
- **Smart default sources with locale boost** — new installations start with a curated subset (~101 sources, Tier 1+2 per panel, minimum 8 per panel) rather than all 150+ feeds. For non-English users, a one-time locale-aware boost automatically enables native-language feeds matching the browser's language (e.g., viewing in Korean enables Yonhap and Chosun Ilbo; viewing in Greek enables Kathimerini, Naftemporiki, and Proto Thema). 17 languages have dedicated native-language feed sets — feeds declare their `lang` field and the boost function matches against the browser locale
|
||||
- **8+ default live video streams** — Bloomberg, Sky News, Al Jazeera, Euronews, DW, France24, CNBC, Al Arabiya — with automatic live detection that scrapes YouTube channel pages every 5 minutes to find active streams. 30+ additional channels available from an expandable library (Fox, BBC, CNN Turk, TRT, RT, CBS, NBC, CNN Brasil, and more) across 6 region tabs including **Oceania** (ABC News Australia)
|
||||
- **HLS native streaming** — 18+ channels (Sky News, Euronews, DW, France24, Al Arabiya, CBS News, TRT World, Sky News Arabia, Al Hadath, RT, ABC News AU, Tagesschau24, India Today, KAN 11, TV5Monde Info, Arise News, NHK World, Fox News, and more) stream via native HLS `<video>` elements instead of YouTube iframes, bypassing cookie popups, bot checks, and WKWebView autoplay restrictions. CNN and CNBC stream via a proxied HLS path through the sidecar. HLS failure triggers automatic 5-minute cooldown with YouTube iframe fallback. RT (Russia Today) — banned from YouTube — streams exclusively via HLS
|
||||
- **Fullscreen live video** — a toggle button in the Live News panel header expands the video grid to fill the entire viewport, hiding all other panels. The fullscreen state applies CSS overrides to both the panel element and the document body, with an icon that swaps between expand and collapse states
|
||||
- **Desktop embed bridge** — YouTube's IFrame API restricts playback in native webviews (error 153). The dashboard detects this and transparently routes through a cloud-hosted embed proxy with bidirectional message passing (play/pause/mute/unmute/loadVideo)
|
||||
- **Idle-aware playback** — video players pause and are removed from the DOM after 5 minutes of inactivity, resuming when the user returns. Tab visibility changes also suspend/resume streams
|
||||
- **Global streaming quality control** — a user-selectable quality setting (auto, 360p, 480p, 720p) that applies to all live video streams across the dashboard. The preference persists in localStorage and propagates to active players via a `stream-quality-changed` CustomEvent — no reload required when switching quality
|
||||
- **22 live webcams** — real-time YouTube streams from geopolitical hotspots across 5 regions (Iran/Attacks, Middle East, Europe, Americas, Asia-Pacific). Grid view shows 4 strategic feeds simultaneously; single-feed view available. Region filtering (IRAN/MIDEAST/EUROPE/AMERICAS/ASIA), idle-aware playback that pauses after 5 minutes, and Intersection Observer-based lazy loading. The Iran/Attacks tab provides a dedicated 2×2 grid of Tehran, Tel Aviv, and Jerusalem feeds for real-time visual monitoring during escalation events
|
||||
- **Custom keyword monitors** — user-defined keyword alerts with word-boundary matching (prevents "ai" from matching "train"), automatic color-coding from a 10-color palette, and multi-keyword support (comma-separated). Monitors search across both headline titles and descriptions and show real-time match counts
|
||||
- **Breaking news click-through** — clicking a breaking news banner scrolls the page to the RSS panel that sourced the alert and applies a 1.5-second flash highlight animation. The source mapping uses `getSourcePanelId()` to resolve each news source name to its parent category panel
|
||||
- **Entity extraction** — Auto-links countries, leaders, organizations
|
||||
- **Instant flat render** — news items appear immediately as a flat list the moment feed data arrives. ML-based clustering (topic grouping, entity extraction, sentiment analysis) runs asynchronously in the background and progressively upgrades the view when ready — eliminating the 1–3 second blank delay that would occur if clustering blocked initial render. Finance variant categories fetch with 5 concurrent requests (vs 3) for ~10–15 second faster cold starts
|
||||
- **Virtual scrolling** — news panels with 15+ items use a custom virtual list renderer that only creates DOM elements for visible items plus a 3-item overscan buffer. Viewport spacers simulate full-list height. Uses `requestAnimationFrame`-batched scroll handling and `ResizeObserver` for responsive adaptation. DOM elements are pooled and recycled rather than created/destroyed
|
||||
@@ -242,6 +250,10 @@ All four variants run from a single codebase — switch between them with one cl
|
||||
- **Browser-side ML worker** (Transformers.js) for NER and sentiment analysis without server dependency — controllable via a "Browser Local Model" toggle in AI Flow settings. When disabled, the ML worker is never initialized, eliminating ONNX model downloads and WebGL memory allocation. The toggle propagates dynamically — enabling it mid-session initializes the worker immediately, disabling it terminates it
|
||||
- **Cmd+K command palette** — fuzzy search across 20+ result types (news, countries, hotspots, markets, bases, cables, datacenters, nuclear facilities, and more), plus layer toggle commands, layer presets (e.g., `layers:military`, `layers:finance`), and instant country brief navigation for all ~250 ISO countries with flag emoji icons. Curated countries include search aliases (e.g., typing "kremlin" or "putin" finds Russia). Scoring ranks exact matches (3pts) above prefix matches (2pts) above substring matches (1pt). Recent searches are stored in localStorage (max 8 entries)
|
||||
- **Historical playback** — dashboard snapshots are stored in IndexedDB. A time slider allows rewinding to any saved state, with live updates paused during playback
|
||||
- **TV Mode** — an ambient fullscreen panel cycling mode designed for the Happy variant (also available on all variants). Entering TV Mode goes fullscreen, hides all panels except one, and cycles through each panel and the map on a configurable timer (30 seconds to 2 minutes, default 1 minute). The interval is persisted in localStorage. Press Escape to exit. Driven by CSS via `[data-tv-mode]` data attribute with visual overrides in `happy-theme.css`
|
||||
- **Badge animation toggle** — an opt-in setting in the unified settings panel ("Badge Animation") enables a CSS `bump` keyframe animation on panel count badges whenever their count increases. Disabled by default to avoid distraction. The Intelligence Findings badge has its own always-on `pulse` animation for new findings
|
||||
- **Cache purge admin endpoint** — `POST /api/cache-purge` allows targeted Redis key deletion without redeploying. Accepts up to 20 explicit keys and/or 3 glob patterns, uses SCAN-based resolution (max 5 iterations, 200 deletion cap), protects rate-limit and durable data prefixes, and requires timing-safe HMAC authentication via `RELAY_SHARED_SECRET`. Supports `dryRun: true` for preview
|
||||
- **Mobile-optimized map** — on mobile devices, the map supports single-finger pan with an 8px movement threshold and inertial velocity animation (0.92 decay factor, velocity computed from 4-entry circular touch history), two-finger pinch-to-zoom with center-point preservation, and a bottom-sheet popup mode that slides up from the bottom with drag-to-dismiss (96px threshold). A click guard prevents accidental popup opens after drag gestures. User location detection uses timezone mapping first (zero network dependency), upgrading to precise geolocation only when the permission is already granted
|
||||
- **Mobile detection** — screens below 768px receive a warning modal since the dashboard is designed for multi-panel desktop use
|
||||
- **UCDP conflict classification** — countries with active wars (1,000+ battle deaths/year) receive automatic CII floor scores, preventing optimistic drift. The UCDP GED API integration uses automatic version discovery (probing multiple year-based API versions in parallel), negative caching (5-minute backoff after upstream failures), discovered-version caching (1-hour TTL), and stale-on-error fallback to ensure conflict data is always available even when the upstream API is intermittently down
|
||||
- **HAPI humanitarian data** — UN OCHA humanitarian access metrics and displacement flows feed into country-level instability scoring with dual-perspective (origins vs. hosts) panel
|
||||
@@ -359,6 +371,52 @@ All three API tiers (Ollama, Groq, OpenRouter) share a common handler factory (`
|
||||
|
||||
The Ollama tier communicates via the OpenAI-compatible `/v1/chat/completions` endpoint, making it compatible with any local inference server that implements this standard (Ollama, LM Studio, llama.cpp server, vLLM, etc.).
|
||||
|
||||
### AI Deduction & Forecasting
|
||||
|
||||
The Deduction Panel is an interactive AI geopolitical analysis tool that produces near-term timeline forecasts grounded in live intelligence data.
|
||||
|
||||
**Request pipeline**:
|
||||
|
||||
1. The analyst enters a free-text query (e.g., "What will happen in the next 24 hours in the Middle East?") and an optional geographic context field
|
||||
2. Before submission, `buildNewsContext()` pulls the 15 most recent `NewsItem` titles from the live feed and prepends them as structured context (`"Recent News:\n- Headline (Source)"`) — ensuring the LLM always has current situational awareness
|
||||
3. The query is sent to the `deductSituation` RPC endpoint, which calls a Groq LLM (or any OpenAI-compatible endpoint via `LLM_API_URL`/`LLM_MODEL` env vars) with a system prompt instructing it to act as a "senior geopolitical intelligence analyst and forecaster"
|
||||
4. Temperature is 0.3 (low, for analytic consistency), max 1,500 tokens. Chain-of-thought `<think>` tags are stripped as defense-in-depth
|
||||
5. Results are cached in Redis for 1 hour by `deduct:situation:v1:<hash(query|geoContext)>` — identical queries serve instantly from cache
|
||||
|
||||
**Cross-panel integration**: Any panel can dispatch a `wm:deduct-context` custom DOM event with `{ query, geoContext, autoSubmit }`, which pre-fills the Deduction Panel and optionally auto-submits. This enables contextual forecasting from any part of the dashboard — clicking "Analyze" on a theater posture card can automatically trigger a regional deduction. A 5-second cooldown prevents rapid re-submission.
|
||||
|
||||
The panel is lazy-loaded (`import()`) to exclude DOMPurify from the main bundle unless the panel is actually accessed, keeping the web bundle lean.
|
||||
|
||||
### Client-Side Headline Memory (RAG)
|
||||
|
||||
The Headline Memory system provides browser-local Retrieval-Augmented Generation — a persistent semantic index of news headlines that runs entirely on the user's device.
|
||||
|
||||
**Ingestion pipeline**:
|
||||
|
||||
```
|
||||
RSS Feed Parse → isHeadlineMemoryEnabled()? → ML Worker (Web Worker)
|
||||
│
|
||||
┌─────────┴──────────┐
|
||||
│ ONNX Embeddings │
|
||||
│ all-MiniLM-L6-v2 │
|
||||
│ 384-dim float32 │
|
||||
└─────────┬──────────┘
|
||||
│
|
||||
┌─────────┴──────────┐
|
||||
│ IndexedDB Store │
|
||||
│ 5,000 vector cap │
|
||||
│ LRU by ingestAt │
|
||||
└────────────────────┘
|
||||
```
|
||||
|
||||
1. After each RSS feed fetch and parse, if Headline Memory is enabled and the embeddings model is loaded, each headline's title, publication date, source, URL, and location tags are sent to the ML Worker
|
||||
2. The worker sanitizes text (strips control chars, truncates to 200 chars), embeds via the ONNX pipeline (`pooling: 'mean', normalize: true`), and deduplicates by content hash
|
||||
3. Vectors are written to IndexedDB via a serialized promise queue (preventing concurrent transaction conflicts). When the 5,000-vector cap is exceeded, the oldest entries by `ingestedAt` are evicted
|
||||
|
||||
**Search**: Queries are embedded using the same model, then a full cursor scan computes cosine similarity against all stored vectors. Results are ranked by score, capped at `topK` (1–20), and filtered by `minScore` (0–1). Multiple query strings can be searched simultaneously (up to 5), with the max score per record across all queries used for ranking.
|
||||
|
||||
**Opt-in mechanism**: The setting defaults to `false` (stored as `wm-headline-memory` in localStorage). Enabling it triggers `mlWorker.init()` → `loadModel('embeddings')`. Disabling it unloads the model and optionally terminates the entire worker if no other ML features are active. The `ai-flow-changed` CustomEvent propagates toggle changes to all interested components.
|
||||
|
||||
### Threat Classification Pipeline
|
||||
|
||||
Every news item passes through a three-stage classification pipeline:
|
||||
@@ -566,7 +624,7 @@ Messages are deduplicated by ID, filtered to exclude media-only posts (images wi
|
||||
|
||||
The dashboard monitors Israel's Home Front Command (Pikud HaOref) alert system for incoming rocket, missile, and drone sirens — a real-time signal that is difficult to obtain programmatically due to Akamai WAF protection.
|
||||
|
||||
**Data flow**: The Railway relay polls `oref.org.il` using `curl` (not Node.js fetch, which is JA3-blocked) through a residential proxy with an Israeli exit IP. On startup, the relay bootstraps 24-hour alert history, then polls live alerts every 5 minutes. Wave detection groups individual siren records by timestamp to identify distinct attack waves. Israel-local timestamps are converted to UTC with DST-aware offset calculation.
|
||||
**Data flow**: The Railway relay polls `oref.org.il` using `curl` (not Node.js fetch, which is JA3-blocked) through a residential proxy with an Israeli exit IP. On startup, the relay bootstraps history via a two-phase strategy: Phase 1 loads from Redis (filtering entries older than 7 days); if Redis is empty, Phase 2 fetches from the upstream OREF API with exponential backoff retry (up to 3 attempts, delays of 3s/6s/12s + jitter). Alert history is persisted to Redis with dirty-flag deduplication to prevent redundant writes. Live alerts are polled every 5 minutes. Wave detection groups individual siren records by timestamp to identify distinct attack waves. Israel-local timestamps are converted to UTC with DST-aware offset calculation. **1,478 Hebrew→English location translations** — an auto-generated dictionary (from the pikud-haoref-api `cities.json` source) enables automatic translation of Hebrew city names in alert data. Unicode bidirectional control characters are stripped via `sanitizeHebrew()` before translation lookups to prevent mismatches.
|
||||
|
||||
**CII integration**: Active OREF alerts boost Israel's CII conflict component by up to 50 points (`25 + min(25, alertCount × 5)`). Rolling 24-hour history adds a secondary boost: 3–9 alerts in the window contribute +5, 10+ contribute +10 to the blended score. This means a sustained multi-wave rocket barrage drives Israel's CII significantly higher than a single isolated alert.
|
||||
|
||||
@@ -806,6 +864,42 @@ The webcam panel supports two viewing modes: a 4-feed grid (default strategic se
|
||||
|
||||
Resource management is aggressive — iframes are lazy-loaded via Intersection Observer (only rendered when the panel scrolls into view), paused after 5 minutes of user inactivity, and destroyed from the DOM entirely when the browser tab is hidden. On Tauri desktop, YouTube embeds route through a cloud proxy to bypass WKWebView autoplay restrictions. Each feed carries a fallback video ID in case the primary stream goes offline.
|
||||
|
||||
### Server-Side Feed Aggregation
|
||||
|
||||
Rather than each client browser independently fetching dozens of RSS feeds through individual edge function invocations, the `listFeedDigest` RPC endpoint aggregates all feeds server-side into a single categorized response.
|
||||
|
||||
**Architecture**:
|
||||
|
||||
```
|
||||
Client (1 RPC call) → listFeedDigest → Redis check (digest:v1:{variant}:{lang})
|
||||
│
|
||||
┌─────────┴─── HIT → return cached digest
|
||||
│
|
||||
▼ MISS
|
||||
┌─────────────────────────┐
|
||||
│ buildDigest() │
|
||||
│ 20 concurrent fetches │
|
||||
│ 8s per-feed timeout │
|
||||
│ 25s overall deadline │
|
||||
└────────┬────────────────┘
|
||||
│
|
||||
┌─────┴─────┐
|
||||
│ Per-feed │ ← cached 600s per URL
|
||||
│ Redis │
|
||||
└─────┬─────┘
|
||||
│
|
||||
▼
|
||||
┌─────────────────────────┐
|
||||
│ Categorized digest │
|
||||
│ Cached 900s (15 min) │
|
||||
│ Per-item keyword class. │
|
||||
└─────────────────────────┘
|
||||
```
|
||||
|
||||
The digest cache key is `news:digest:v1:{variant}:{lang}` with a 900-second TTL. Individual feed results are separately cached per URL for 600 seconds. Items per feed are capped at 5, categories at 20 items each. XML parsing is edge-runtime-compatible (regex-based, no DOM parser), handling both RSS `<item>` and Atom `<entry>` formats. Each item is keyword-classified at aggregation time. An in-memory fallback cache (capped at 50 entries) provides last-known-good data if Redis fails.
|
||||
|
||||
This eliminates per-client feed fan-out — 1,000 concurrent users each polling 25 feed categories would have generated 25,000 edge invocations per poll cycle. With server-side aggregation, they generate exactly 1 (or 0 if the digest is cached).
|
||||
|
||||
### Desktop Auto-Update
|
||||
|
||||
The desktop app checks for new versions by polling `worldmonitor.app/api/version` — once at startup (after a 5-second delay) and then every 6 hours. When a newer version is detected (semver comparison), a non-intrusive update badge appears with a direct link to the GitHub Release page.
|
||||
@@ -1159,7 +1253,7 @@ All three variants run on three platforms that work together:
|
||||
- **OREF rocket alerts** — polls Israel's Home Front Command alert system via `curl` through a residential proxy (Akamai WAF blocks datacenter TLS fingerprints)
|
||||
- **Polymarket proxy** — fetches from Gamma API with concurrent upstream limiting (max 3 simultaneous, queue backpressure at 20), in-flight deduplication, and 10-minute caching to prevent stampedes from 11 parallel tag queries
|
||||
- **ICAO NOTAM proxy** — routes NOTAM closure queries through the relay for MENA airports, bypassing Vercel IP restrictions on ICAO's API
|
||||
- **RSS feeds** — proxies feeds from domains that block Vercel IPs, with a separate domain allowlist for security
|
||||
- **RSS feeds** — proxies feeds from domains that block Vercel IPs, with a separate domain allowlist for security. Supports conditional GET (ETag/If-Modified-Since) to reduce bandwidth for unchanged feeds
|
||||
|
||||
The Vercel edge functions connect to Railway via `WS_RELAY_URL` (server-side, HTTPS) while browser clients connect via `VITE_WS_RELAY_URL` (client-side, WSS). This separation keeps the relay URL configurable per deployment without leaking server-side configuration to the browser.
|
||||
|
||||
@@ -1492,18 +1586,18 @@ Set `WS_RELAY_URL` (server-side, HTTPS) and `VITE_WS_RELAY_URL` (client-side, WS
|
||||
| --------------------- | ---------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| **Frontend** | TypeScript, Vite, deck.gl (WebGL 3D globe), MapLibre GL, vite-plugin-pwa (service worker + manifest) |
|
||||
| **Desktop** | Tauri 2 (Rust) with Node.js sidecar, OS keychain integration (keyring crate), native TLS (reqwest) |
|
||||
| **AI/ML** | Ollama / LM Studio (local, OpenAI-compatible), Groq (Llama 3.1 8B), OpenRouter (fallback), Transformers.js (browser-side T5, NER, embeddings) |
|
||||
| **AI/ML** | Ollama / LM Studio (local, OpenAI-compatible), Groq (Llama 3.1 8B), OpenRouter (fallback), Transformers.js (browser-side T5, NER, embeddings), IndexedDB vector store (5K headline RAG) |
|
||||
| **Caching** | Redis (Upstash) — 3-tier cache with in-memory + Redis + upstream, cross-user AI deduplication. Vercel CDN (s-maxage). Service worker (Workbox) |
|
||||
| **Geopolitical APIs** | OpenSky, GDELT, ACLED, UCDP, HAPI, USGS, GDACS, NASA EONET, NASA FIRMS, Polymarket, Cloudflare Radar, WorldPop, OREF (Israel sirens), gpsjam.org (GPS interference), Telegram MTProto (27 OSINT channels) |
|
||||
| **Market APIs** | Yahoo Finance (equities, forex, crypto), CoinGecko (stablecoins), mempool.space (BTC hashrate), alternative.me (Fear & Greed) |
|
||||
| **Threat Intel APIs** | abuse.ch (Feodo Tracker, URLhaus), AlienVault OTX, AbuseIPDB, C2IntelFeeds |
|
||||
| **Economic APIs** | FRED (Federal Reserve), EIA (Energy), Finnhub (stock quotes) |
|
||||
| **Localization** | i18next (16 languages: en, fr, de, es, it, pl, pt, nl, sv, ru, ar, zh, ja, tr, th, vi), RTL support, lazy-loaded bundles, native-language feeds for 7 locales |
|
||||
| **Localization** | i18next (16 languages: en, fr, de, es, it, pl, pt, nl, sv, ru, ar, zh, ja, tr, th, vi), RTL support, lazy-loaded bundles, native-language feeds for 17 locales with one-time locale boost |
|
||||
| **API Contracts** | Protocol Buffers (92 proto files, 20 services), sebuf HTTP annotations, buf CLI (lint + breaking checks), auto-generated TypeScript clients/servers + OpenAPI 3.1.0 docs |
|
||||
| **Analytics** | PostHog (privacy-first, typed event schemas, pseudonymous identity, ad-blocker bypass via reverse proxy, offline queue for desktop) |
|
||||
| **Analytics** | Vercel Analytics (privacy-first, lightweight web vitals and page view tracking) |
|
||||
| **Deployment** | Vercel Edge Functions (60+ endpoints) + Railway (WebSocket relay + Telegram + OREF + Polymarket proxy + NOTAM) + Tauri (macOS/Windows/Linux) + PWA (installable) |
|
||||
| **Finance Data** | 92 stock exchanges, 19 financial centers, 13 central banks, 10 commodity hubs, 64 Gulf FDI investments |
|
||||
| **Data** | 150+ RSS feeds, ADS-B transponders, AIS maritime data, VIIRS satellite imagery, 30+ live video channels (8+ default YouTube + 10 HLS native), 27 Telegram OSINT channels |
|
||||
| **Data** | 150+ RSS feeds, ADS-B transponders, AIS maritime data, VIIRS satellite imagery, 30+ live video channels (8+ default YouTube + 18+ HLS native), 27 Telegram OSINT channels |
|
||||
|
||||
---
|
||||
|
||||
@@ -1648,6 +1742,26 @@ Desktop release details, signing hooks, variant outputs, and clean-machine valid
|
||||
- [x] Strategic Risk Score algorithm (composite geopolitical risk from convergence, CII, infrastructure, theater, breaking news)
|
||||
- [x] HLS native streaming (10 channels bypass YouTube iframes — Sky News, Euronews, DW, France24, RT, and more)
|
||||
- [x] Polymarket in-flight request deduplication and queue backpressure (max 20 queued, response slicing)
|
||||
- [x] AI Deduction & Forecasting panel (LLM-powered near-term geopolitical timeline analysis with live headline context)
|
||||
- [x] Headline Memory (RAG) — opt-in client-side vector store with ONNX embeddings, IndexedDB persistence, and semantic search
|
||||
- [x] Server-side feed aggregation via `listFeedDigest` RPC — reduces Vercel Edge invocations by ~95%
|
||||
- [x] Smart default source reduction (~101 from 150+) with one-time locale-aware boost for 17 languages
|
||||
- [x] Gulf Economies panel (GCC indices, currencies, oil with mini sparklines and 60-second polling)
|
||||
- [x] Mobile-optimized map (touch pan with inertia, pinch-to-zoom, bottom-sheet popups, timezone-based location detection)
|
||||
- [x] 18+ HLS native streaming channels (Fox News, ABC News AU, NHK World, TV5Monde, Tagesschau24, and more)
|
||||
- [x] Live video fullscreen toggle (expand video grid to fill viewport)
|
||||
- [x] Breaking news click-through (scroll to source panel with flash highlight)
|
||||
- [x] TV Mode (ambient fullscreen panel cycling with configurable interval, 30s–2min)
|
||||
- [x] Badge animation toggle (opt-in bump animation on panel count changes)
|
||||
- [x] Cache purge admin endpoint (targeted Redis key deletion with HMAC auth and dry-run mode)
|
||||
- [x] OREF Redis persistence with two-phase bootstrap (Redis-first, upstream-fallback with exponential retry)
|
||||
- [x] 1,478 Hebrew→English OREF location translations with Unicode sanitization
|
||||
- [x] Oceania region tab for live channels (ABC News Australia)
|
||||
- [x] PostHog removal (replaced with lightweight Vercel Analytics)
|
||||
- [x] Conditional GET (ETag/If-Modified-Since) on Railway relay RSS feeds
|
||||
- [x] Mobile responsiveness improvements (collapsible map, panel sizing, font scaling)
|
||||
- [x] PWA service worker update fix (stops auto-reload on SW update)
|
||||
- [x] Lazy-loaded DeductionPanel (excludes DOMPurify from web bundle unless panel is accessed)
|
||||
- [ ] Mobile-optimized views
|
||||
- [ ] Push notifications for critical alerts
|
||||
- [ ] Self-hosted Docker image
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
{
|
||||
"name": "world-monitor",
|
||||
"private": true,
|
||||
"version": "2.5.21",
|
||||
"version": "2.5.22",
|
||||
"license": "AGPL-3.0-only",
|
||||
"type": "module",
|
||||
"scripts": {
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "world-monitor"
|
||||
version = "2.5.21"
|
||||
version = "2.5.22"
|
||||
description = "World Monitor desktop application"
|
||||
authors = ["World Monitor"]
|
||||
edition = "2021"
|
||||
|
||||
@@ -2,7 +2,7 @@
|
||||
"$schema": "https://schema.tauri.app/config/2",
|
||||
"productName": "World Monitor",
|
||||
"mainBinaryName": "world-monitor",
|
||||
"version": "2.5.21",
|
||||
"version": "2.5.22",
|
||||
"identifier": "app.worldmonitor.desktop",
|
||||
"build": {
|
||||
"beforeDevCommand": "npm run build:sidecar-sebuf && npm run dev",
|
||||
|
||||
Reference in New Issue
Block a user