eliott/worldmonitor
1
0
Fork 0
mirror of https://github.com/koala73/worldmonitor.git synced 2026-04-25 17:14:57 +02:00
Code Issues Packages Projects Releases Wiki Activity

perf(rss): route RSS direct to Railway, skip Vercel middleman (#961)

Browse Source 
* perf(rss): route RSS direct to Railway, skip Vercel middleman

Vercel /api/rss-proxy has 65% error rate (207K failed invocations/12h).
Route browser RSS requests directly to Railway (proxy.worldmonitor.app)
via Cloudflare CDN, eliminating Vercel as middleman.

- Add VITE_RSS_DIRECT_TO_RELAY feature flag (default off) for staged rollout
- Centralize RSS proxy URL in rssProxyUrl() with desktop/dev/prod routing
- Make Railway /rss public (skip auth, keep rate limiting with CF-Connecting-IP)
- Add wildcard *.worldmonitor.app CORS + always emit Vary: Origin on /rss
- Extract ~290 RSS domains to shared/rss-allowed-domains.cjs (single source of truth)
- Convert Railway domain check to Set for O(1) lookups
- Remove rss-proxy from KEYED_CLOUD_API_PATTERN (no longer needs API key header)
- Add edge function test for shared domain list import

* fix(edge): replace node:module with JSON import for edge-compatible RSS domains

api/_rss-allowed-domains.js used createRequire from node:module which is
unsupported in Vercel Edge Runtime, breaking all edge functions (including
api/gpsjam). Replaced with JSON import attribute syntax that works in both
esbuild (Vercel build) and Node.js 22+ (tests).

Also fixed middleware.ts TS18048 error where VARIANT_OG[variant] could be
undefined.

* test(edge): add guard against node: built-in imports in api/ files

Scans ALL api/*.js files (including _ helpers) for node: module imports
which are unsupported in Vercel Edge Runtime. This would have caught the
createRequire(node:module) bug before it reached Vercel.

* fix(edge): inline domain array and remove NextResponse reference

- Replace `import ... with { type: 'json' }` in _rss-allowed-domains.js
  with inline array — Vercel esbuild doesn't support import attributes
- Replace `NextResponse.next()` with bare `return` in middleware.ts —
  NextResponse was never imported

* ci(pre-push): add esbuild bundle check and edge function tests

The pre-push hook now catches Vercel build failures locally:
- esbuild bundles each api/*.js entrypoint (catches import attribute
  syntax, missing modules, and other bundler errors)
- runs edge function test suite (node: imports, module isolation)
This commit is contained in:
Elie Habib
2026-03-04 18:42:00 +04:00
committed by GitHub
parent 8fd732be1f
commit 898ac7b1c4
15 changed files with 640 additions and 339 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
shared/rss-allowed-domains.cjs Normal file
View File

@@ -0,0 +1,2 @@
// CJS wrapper — source of truth is rss-allowed-domains.json
module.exports = require('./rss-allowed-domains.json');

266
shared/rss-allowed-domains.json Normal file
View File

@@ -0,0 +1,266 @@
[
"feeds.bbci.co.uk",
"www.theguardian.com",
"feeds.npr.org",
"news.google.com",
"www.aljazeera.com",
"www.aljazeera.net",
"rss.cnn.com",
"hnrss.org",
"feeds.arstechnica.com",
"www.theverge.com",
"www.cnbc.com",
"feeds.marketwatch.com",
"www.defenseone.com",
"breakingdefense.com",
"www.bellingcat.com",
"techcrunch.com",
"huggingface.co",
"www.technologyreview.com",
"rss.arxiv.org",
"export.arxiv.org",
"www.federalreserve.gov",
"www.sec.gov",
"www.whitehouse.gov",
"www.state.gov",
"www.defense.gov",
"home.treasury.gov",
"www.justice.gov",
"tools.cdc.gov",
"www.fema.gov",
"www.dhs.gov",
"www.thedrive.com",
"krebsonsecurity.com",
"finance.yahoo.com",
"thediplomat.com",
"venturebeat.com",
"foreignpolicy.com",
"www.ft.com",
"openai.com",
"www.reutersagency.com",
"feeds.reuters.com",
"rsshub.app",
"asia.nikkei.com",
"www.cfr.org",
"www.csis.org",
"www.politico.com",
"www.brookings.edu",
"layoffs.fyi",
"www.defensenews.com",
"www.militarytimes.com",
"taskandpurpose.com",
"news.usni.org",
"www.oryxspioenkop.com",
"www.gov.uk",
"www.foreignaffairs.com",
"www.atlanticcouncil.org",
"www.zdnet.com",
"www.techmeme.com",
"www.darkreading.com",
"www.schneier.com",
"www.ransomware.live",
"rss.politico.com",
"www.anandtech.com",
"www.tomshardware.com",
"www.semianalysis.com",
"feed.infoq.com",
"thenewstack.io",
"devops.com",
"dev.to",
"lobste.rs",
"changelog.com",
"seekingalpha.com",
"news.crunchbase.com",
"www.saastr.com",
"feeds.feedburner.com",
"www.producthunt.com",
"www.axios.com",
"api.axios.com",
"github.blog",
"githubnext.com",
"mshibanami.github.io",
"www.engadget.com",
"news.mit.edu",
"dev.events",
"www.ycombinator.com",
"a16z.com",
"review.firstround.com",
"www.sequoiacap.com",
"www.nfx.com",
"www.aaronsw.com",
"bothsidesofthetable.com",
"www.lennysnewsletter.com",
"stratechery.com",
"www.eu-startups.com",
"tech.eu",
"sifted.eu",
"www.techinasia.com",
"kr-asia.com",
"techcabal.com",
"disrupt-africa.com",
"lavca.org",
"contxto.com",
"inc42.com",
"yourstory.com",
"pitchbook.com",
"www.cbinsights.com",
"www.techstars.com",
"asharqbusiness.com",
"asharq.com",
"www.omanobserver.om",
"english.alarabiya.net",
"www.arabnews.com",
"www.timesofisrael.com",
"www.haaretz.com",
"www.scmp.com",
"kyivindependent.com",
"www.themoscowtimes.com",
"feeds.24.com",
"feeds.news24.com",
"feeds.capi24.com",
"www.france24.com",
"www.euronews.com",
"de.euronews.com",
"es.euronews.com",
"fr.euronews.com",
"it.euronews.com",
"pt.euronews.com",
"ru.euronews.com",
"www.lemonde.fr",
"rss.dw.com",
"www.bild.de",
"www.africanews.com",
"fr.africanews.com",
"www.premiumtimesng.com",
"www.vanguardngr.com",
"www.channelstv.com",
"dailytrust.com",
"www.thisdaylive.com",
"www.naftemporiki.gr",
"www.in.gr",
"www.iefimerida.gr",
"www.lasillavacia.com",
"www.channelnewsasia.com",
"japantoday.com",
"www.thehindu.com",
"indianexpress.com",
"www.twz.com",
"gcaptain.com",
"news.un.org",
"www.iaea.org",
"www.who.int",
"www.cisa.gov",
"www.crisisgroup.org",
"rusi.org",
"warontherocks.com",
"www.aei.org",
"responsiblestatecraft.org",
"www.fpri.org",
"jamestown.org",
"www.chathamhouse.org",
"ecfr.eu",
"www.gmfus.org",
"www.wilsoncenter.org",
"www.lowyinstitute.org",
"www.mei.edu",
"www.stimson.org",
"www.cnas.org",
"carnegieendowment.org",
"www.rand.org",
"fas.org",
"www.armscontrol.org",
"www.nti.org",
"thebulletin.org",
"www.iss.europa.eu",
"www.fao.org",
"worldbank.org",
"www.imf.org",
"www.bbc.com",
"www.spiegel.de",
"www.tagesschau.de",
"newsfeed.zeit.de",
"feeds.elpais.com",
"e00-elmundo.uecdn.es",
"www.repubblica.it",
"www.ansa.it",
"xml2.corriereobjects.it",
"feeds.nos.nl",
"www.nrc.nl",
"www.telegraaf.nl",
"www.dn.se",
"www.svd.se",
"www.svt.se",
"www.asahi.com",
"www.clarin.com",
"oglobo.globo.com",
"feeds.folha.uol.com.br",
"www.eltiempo.com",
"www.eluniversal.com.mx",
"www.jeuneafrique.com",
"www.lorientlejour.com",
"www.hurriyet.com.tr",
"tvn24.pl",
"www.polsatnews.pl",
"www.rp.pl",
"meduza.io",
"novayagazeta.eu",
"www.bangkokpost.com",
"vnexpress.net",
"www.abc.net.au",
"islandtimes.org",
"www.brasilparalelo.com.br",
"mexiconewsdaily.com",
"insightcrime.org",
"www.primicias.ec",
"www.infobae.com",
"www.eluniverso.com",
"news.ycombinator.com",
"www.coindesk.com",
"cointelegraph.com",
"travel.state.gov",
"smartraveller.gov.au",
"www.smartraveller.gov.au",
"www.safetravel.govt.nz",
"th.usembassy.gov",
"ae.usembassy.gov",
"de.usembassy.gov",
"ua.usembassy.gov",
"mx.usembassy.gov",
"in.usembassy.gov",
"pk.usembassy.gov",
"co.usembassy.gov",
"pl.usembassy.gov",
"bd.usembassy.gov",
"it.usembassy.gov",
"do.usembassy.gov",
"mm.usembassy.gov",
"wwwnc.cdc.gov",
"www.ecdc.europa.eu",
"www.afro.who.int",
"www.goodnewsnetwork.org",
"www.positive.news",
"reasonstobecheerful.world",
"www.optimistdaily.com",
"www.upworthy.com",
"www.dailygood.org",
"www.goodgoodgood.co",
"www.good.is",
"www.sunnyskyz.com",
"thebetterindia.com",
"singularityhub.com",
"humanprogress.org",
"greatergood.berkeley.edu",
"www.onlygoodnewsdaily.com",
"www.sciencedaily.com",
"feeds.nature.com",
"www.nature.com",
"www.livescience.com",
"www.newscientist.com",
"www.pbs.org",
"feeds.abcnews.com",
"feeds.nbcnews.com",
"www.cbsnews.com",
"moxie.foxnews.com",
"feeds.content.dowjones.io",
"thehill.com"
]