mirror of
https://github.com/koala73/worldmonitor.git
synced 2026-04-25 17:14:57 +02:00
6d66c06f07
* fix(consumer-prices): harden scrape/aggregate/publish pipeline - scrape: treat 0-product parse as error (increments errorsCount, skips pagesSucceeded) so noon_grocery_ae missing eggs_12/tomatoes_1kg marks the run partial instead of completed - publish: fix freshData gate (freshnessMin >= 0) so a scrape finishing at exactly 0 min lag still advances seed-meta - aggregate: wrap per-basket aggregation in try/catch so one failing basket does not skip remaining baskets; re-throw if any failed - seed-consumer-prices.mjs: require --force flag to prevent accidentally stomping publish.ts 26h TTLs with short 10-60min fallback TTLs * fix(consumer-prices): correct basket comparison with intersection + dedup Both aggregate.ts and the retailer spread snapshot were summing ALL matched SKUs per retailer without deduplication, making Carrefour appear most expensive simply because it had more matched products (31 "items" vs Noon's 20 for a 12-item basket). Fixes: - aggregate.ts retailer_spread_pct: deduplicate per (retailer, basketItem) taking cheapest price, then only compare on items all retailers carry - worldmonitor.ts buildRetailerSpreadSnapshot: same dedup + intersection logic in SQL — one best_price per (retailer, basket_item), common_items CTE filters to items every active retailer covers - exa-search.ts parseListing: log whether Exa returned 0 results or results with no extractable price, to distinguish the two failure modes * fix(consumer-prices-panel): correct parse rate display, category names, and freshness colors - parseSuccessRate is stored as 0-100 but UI was doing *100 again (shows 10000%) - Category name builder converts snake_case to Title Case (Cooking_oil → Cooking Oil) - Add missing cp-fresh--ok/warn/stale/unknown CSS classes (freshness labels had no color) - Add border-radius to stat cards and range buttons; add font-family to range buttons - Add padding + bottom border to cp-range-bar for visual separation * fix(consumer-prices): gate overview spread_pct query to last 2 days buildOverviewSnapshot queried retailer_spread_pct with no recency filter, so ORDER BY metric_date DESC LIMIT 1 would serve an arbitrarily old row when today's aggregate run omitted a write (no retailer intersection). Add INTERVAL '2 days' cutoff — covers 24h cron cadence plus scheduling drift. Falls through to 0 (→ UI shows '—') when no recent value exists.