Commit Graph

3 Commits

Author SHA1 Message Date
Elie Habib
cbb51a887a fix(seeds): re-add military-bases to static-ref bundle with 10min timeout (#2895)
The script processes 125K records and needs ~3-4 min. The previous 120s
timeout was too short. Increased to 600s (10min) to allow completion.
2026-04-10 14:23:48 +04:00
Elie Habib
d85bee163e fix(seeds): remove military-bases from static-ref bundle (#2893)
* fix(seeds): remove military-bases from static-ref bundle

seed-military-bases.mjs is a one-time batch upload tool requiring
--env and --sha CLI args. Without args it exits code 1. It writes
no seed-meta so the freshness gate always tries to run it, always
fails, causing the bundle to exit non-zero.

* fix(notifications): wrap Telegram/Slack/Discord sends in try/catch

Unhandled fetch timeout (ETIMEDOUT) in sendTelegram crashed the entire
digest cron run. Email delivery succeeded but subsequent Telegram
delivery threw an uncaught error, killing the process.

Wrapped all three webhook-style send functions (Telegram, Slack, Discord)
in try/catch so network timeouts log a warning and return false instead
of crashing. sendEmail already had this pattern.
2026-04-10 14:10:30 +04:00
Elie Habib
6d923108d8 refactor(seeds): bundle orchestrator to consolidate Railway cron services (100→65) (#2891)
* refactor(seeds): add bundle orchestrator to consolidate Railway cron services

Railway is at the 100-service limit. This adds a shared _bundle-runner.mjs
orchestrator and 11 bundle scripts that group related seed cron services,
reducing the count from 100 to ~65 when deployed.

Each bundle spawns sub-seeds via child_process.execFile (proven pattern from
ais-relay.cjs), with freshness-gated skipping so monthly seeds in a daily
bundle only run when due. Original scripts are unchanged and independently
runnable.

Bundles: ecb-eu (4→1), portwatch (4→1), climate (5→1), energy-sources (6→1),
macro (6→1), health (4→1), static-ref (3→1), resilience (2→1),
derived-signals (2→1), market-backup (5→1), relay-backup (4→1).

* refactor(seeds): deduplicate time constants across bundle scripts

Export MIN/HOUR/DAY/WEEK from _bundle-runner.mjs so all 11 bundle
scripts import shared constants instead of re-declaring them locally.
Eliminates inconsistent computation styles (24*60*60*1000 vs 24*HOUR).

* fix(seeds): correct wb-indicators seedMetaKey in relay-backup bundle

The seed writes to seed-meta:economic:worldbank-techreadiness:v1 but the
bundle config was missing the :v1 suffix, causing the freshness gate to
always return null and the seed to run every cycle instead of daily.

Found by architecture-strategist review agent.

* fix(seeds): address review findings in bundle runner

- Remove em dashes from comment and log line (project convention)
- Read Redis creds directly instead of via getRedisCredentials() which
  calls process.exit(1) on missing env vars, bypassing try/catch and
  silently killing the entire bundle before any seed runs
- Missing creds now gracefully skip freshness check (seeds still run)

* fix(seeds): correct intervalMs values and exit code in bundle runner

P1 fixes from external review:

1. process.exit(0) on failure now exits non-zero (exit 1 when failed > 0)
   so Railway cron monitoring detects degraded runs.

2. Corrected intervalMs to match actual cron cadences (was using TTL values):
   - crypto-quotes: 15min -> 5min (actual cron is 5min)
   - stablecoin-markets: 15min -> 10min (actual cron is 10min)
   - gulf-quotes: 15min -> 10min (actual cron is 10min)
   - health-air-quality: 3h -> 1h (actual cron is 1h)
   - bls-series: 3d -> 1d (actual cron is daily)
   - eurostat: 3d -> 1d (actual cron is daily)
   - fao-ffpi: 30d -> 1d (runs daily to catch monthly release window)
   - imf-macro: 35d -> 30d (monthly data)
   - national-debt: 35d -> 30d (monthly data)

* docs: add Railway seed consolidation runbook

Complete migration checklist with:
- 46 services to delete (with Railway UUIDs)
- 11 bundle services to create (with cron, start cmd, watch paths)
- 43 standalone services that stay (with reasons)
- Execution order, verification checklist, env var guidance
- Watch paths: scripts/** + shared/** (covers loadSharedConfig resolution)
- Inventory checksum: 4+4+3+46+43 = 100
2026-04-10 11:50:32 +04:00