Files
worldmonitor/scripts/import-bounced-emails.mjs
Elie Habib 20c65a4f4f feat(email): add deliverability guards to reduce waitlist bounces (#2819)
* feat(email): add deliverability guards to reduce waitlist bounces

Analyzed 1,276 bounced waitlist emails and found typos (gamil.com),
disposable domains (passmail, guerrillamail), offensive submissions,
and non-existent domains account for the majority.

Four layers of protection:
- Frontend: mailcheck.js typo suggestions on email blur
- API: MX record check via Cloudflare DoH, disposable domain
  blocklist, offensive pattern filter, typo-TLD blocklist
- Webhook: api/resend-webhook.js captures bounce/complaint events,
  stores in Convex emailSuppressions table, checked before sending
- Tooling: import script for bulk-loading existing bounced emails

* fix(email): address review - auth, retry, CSV parsing

1. Security: Convert suppress/bulkSuppress/remove to internalMutation.
   Webhook now runs as Convex httpAction (matching Dodo pattern) with
   direct access to internal mutations. Bulk import uses relay shared
   secret. Only isEmailSuppressed remains public (read-only query).

2. Retry: Convex httpAction returns 500 on any mutation failure so
   Resend retries the webhook instead of silently losing bounce events.

3. CSV: Replace naive comma-split with RFC 4180 parser that handles
   quoted fields. Import script now calls Convex HTTP action
   authenticated via RELAY_SHARED_SECRET instead of public mutation.

* fix(email): make isEmailSuppressed internal, check inside mutation

Move suppression check into registerInterest:register mutation
(same transaction, no extra round-trip). Remove public query
entirely so no suppression data is exposed to browser clients.

* test(email): add coverage for validation, CSV parser, and suppressions

- 19 tests for validateEmail: disposable domains, offensive patterns,
  typo TLDs, MX fail-open, case insensitivity, privacy relay allowance
- 7 tests for parseCsvLine: RFC 4180 quoting, escaped quotes, empty
  fields, Resend CSV format with angle brackets and commas
- 11 Convex tests for emailSuppressions: suppress idempotency, case
  normalization, bulk dedup, remove, and registerInterest integration
  (emailSuppressed flag in mutation return)
2026-04-08 11:21:40 +04:00

118 lines
3.2 KiB
JavaScript

#!/usr/bin/env node
/**
* One-time script to import bounced emails from a Resend CSV export
* into the Convex emailSuppressions table via the authenticated
* /relay/bulk-suppress-emails HTTP action.
*
* Usage:
* CONVEX_SITE_URL=<your-convex-site-url> RELAY_SHARED_SECRET=<secret> \
* node scripts/import-bounced-emails.mjs <csv-path>
*
* The CSV must have headers including "to" and "last_event".
* Only rows with last_event=bounced are imported.
*/
import { readFileSync } from 'node:fs';
const CONVEX_SITE_URL = process.env.CONVEX_SITE_URL;
const RELAY_SECRET = process.env.RELAY_SHARED_SECRET;
if (!CONVEX_SITE_URL) {
console.error('CONVEX_SITE_URL env var required (e.g. https://your-app.convex.site)');
process.exit(1);
}
if (!RELAY_SECRET) {
console.error('RELAY_SHARED_SECRET env var required');
process.exit(1);
}
const csvPath = process.argv[2];
if (!csvPath) {
console.error('Usage: node scripts/import-bounced-emails.mjs <csv-path>');
process.exit(1);
}
function parseCsvLine(line) {
const fields = [];
let current = '';
let inQuotes = false;
for (let i = 0; i < line.length; i++) {
const ch = line[i];
if (inQuotes) {
if (ch === '"' && line[i + 1] === '"') {
current += '"';
i++;
} else if (ch === '"') {
inQuotes = false;
} else {
current += ch;
}
} else if (ch === '"') {
inQuotes = true;
} else if (ch === ',') {
fields.push(current);
current = '';
} else {
current += ch;
}
}
fields.push(current);
return fields;
}
const raw = readFileSync(csvPath, 'utf-8');
const lines = raw.split('\n').filter(Boolean);
const header = parseCsvLine(lines[0]);
const toIdx = header.indexOf('to');
const eventIdx = header.indexOf('last_event');
if (toIdx === -1 || eventIdx === -1) {
console.error('CSV must have "to" and "last_event" columns');
console.error('Found columns:', header.join(', '));
process.exit(1);
}
const bouncedEmails = [];
for (let i = 1; i < lines.length; i++) {
const cols = parseCsvLine(lines[i]);
if (cols[eventIdx] === 'bounced' && cols[toIdx]) {
bouncedEmails.push(cols[toIdx].trim().toLowerCase());
}
}
const unique = [...new Set(bouncedEmails)];
console.log(`Found ${unique.length} unique bounced emails from ${lines.length - 1} rows`);
const BATCH_SIZE = 100;
let totalAdded = 0;
let totalSkipped = 0;
for (let i = 0; i < unique.length; i += BATCH_SIZE) {
const batch = unique.slice(i, i + BATCH_SIZE).map(email => ({
email,
reason: 'bounce',
source: 'csv-import-2026-04',
}));
const res = await fetch(`${CONVEX_SITE_URL}/relay/bulk-suppress-emails`, {
method: 'POST',
headers: {
'Content-Type': 'application/json',
'Authorization': `Bearer ${RELAY_SECRET}`,
},
body: JSON.stringify({ emails: batch }),
});
if (!res.ok) {
const body = await res.text();
console.error(`Batch ${Math.floor(i / BATCH_SIZE) + 1} failed (${res.status}): ${body}`);
process.exit(1);
}
const result = await res.json();
totalAdded += result.added;
totalSkipped += result.skipped;
console.log(`Batch ${Math.floor(i / BATCH_SIZE) + 1}: +${result.added} added, ${result.skipped} skipped`);
}
console.log(`\nDone: ${totalAdded} added, ${totalSkipped} already suppressed`);