2 Commits

Author SHA1 Message Date
Elie Habib
b92b22bd20 fix(fuel-prices): tolerate Brazil ANP failure to stop Railway crash-loop (#3085)
* fix(fuel-prices): tolerate Brazil ANP failure to stop Railway crash-loop

Brazil gov.br is structurally unreachable from Railway IPs:
- Decodo proxy 403s all .gov.br CONNECTs by policy
- Direct fetch fails undici TLS handshake from Railway egress

After PR #3082 tightened the publish gate to require zero failed sources,
every run exits 1 -> Railway "Deployment crashed" banner + STALE_SEED.

Add TOLERATED_FAILURES = {'Brazil'}; validateFuel ignores tolerated names
when checking failedSources. Critical regions (US/GB/MY) and the >=30
country floor still gate publish. Brazil's outage stays visible via the
existing [FRESHNESS] log.

* fix(fuel-prices): rotate :prev on tolerated-only failures to keep WoW fresh

Reviewer catch: after tolerating Brazil, allSourcesFresh stays false forever
→ :prev never rotates → panel's WoW stretches into 2-week, 3-week, ... deltas
for every non-Brazil country while still labeled 'week-over-week'.

Gate :prev rotation on untolerated failures only. Tolerated sources are
absent from the snapshot entirely, so rotating is safe (no stale-self-
compare poisoning next week).

* fix(fuel-prices): distinguish tolerated vs untolerated sources in [DEGRADED] log

Greptile P2: the [DEGRADED] message said 'publish will be rejected' even
when only tolerated sources (Brazil) failed — confusing for operators
watching Railway logs.
2026-04-14 12:36:07 +04:00
Elie Habib
7e7ca70faf fix(fuel-prices): resilient seeder — proxy, retry, stale-carry-forward, strict gate (#3082)
* fix(fuel-prices): resilient seeder — proxy, retry, stale-carry-forward, strict gate

Addresses 2026-04-07 run where 4 of 7 sources failed (NZ 403, BR/MX fetch failed)
and the seeder silently published 30 countries with Brazil/Mexico/NZ vanishing
from the UI.

- Startup proxy diagnostic so PROXY_URL misconfigs are immediately visible.
- New fetchWithProxyPreferred (proxy-first, direct fallback) + withFuelRetry
  (3 attempts, backoff) wrapping NZ/BR/MX upstream calls.
- Swap MX from dead datos.gob.mx to CRE publicacionexterna XML (13k stations).
- Stale-carry-forward failed sources from :prev snapshot (stale: true) instead
  of dropping countries; fresh-only ranking; skip WoW for stale entries.
- Gate :prev rotation on all-sources-succeeded so partial runs don't poison
  next week's WoW.
- Strict validateFn: >=25 countries AND US+GB+MY fresh. Prior gate was >=1.
- emptyDataIsFailure: true so validation fail doesnt refresh seed-meta.
- Wrap imperative body in main() + isMain guard; export parseCREStationPrices
  and validateFuel; 9 new unit tests.

* fix(fuel-prices): remove stale-carry-forward, harden validator (PR review)

Reviewer flagged two P1s on the prior commit:

1. stale-carry-forward inserted stale: true rows into the published payload,
   but the proto schema and panel have no staleness render path. Users would
   see week-old BR/MX/NZ prices as current. Resilience turned into a
   freshness bug.
2. Validator counted stale-carried entries toward the floor. US/GB/MY fresh
   + 22 stale still passed, refreshing seed-meta.fetchedAt and leaving health
   operationally healthy indefinitely. Hid the outage.

Fix: remove stale-carry-forward entirely. Tighten validator to require
countries.length >= 30, US+GB+MY present, and failedSources.length === 0.
Partial-failure runs now rejected → 10-day cache TTL serves last healthy
snapshot → health STALE_SEED after maxStaleMin. Correct, visible signal.

Drops dead code: SOURCE_COUNTRY_CODES, staleCarried/freshCountries, stale
WoW skip. Tests updated for the failedSources gate.
2026-04-14 09:16:23 +04:00