Automating Certificate Delivery: Durable Queues, Exponential Backoff & Provider Failover

Why Bulk Email Blasts Fail on Campus

When an organizer clicks "Send 1,500 Certificates" using a simple script: University mail filters detect 1,500 incoming emails from the same IP address in 30 seconds. The college firewall flags the IP as a phishing attack and blacklists it. * Half the students never receive their certificates, and the organizer's domain reputation is destroyed.

---

1. The Spark Durable Queue Architecture

In Spark (email_deliveries table and workflow-worker Edge Function): Email dispatches are never executed synchronously inside the HTTP request. Instead, jobs are persisted into a durable PostgreSQL queue (public.email_deliveries).

-- Atomic Worker Job Claiming
SELECT id, recipient, payload
FROM public.email_deliveries
WHERE status = 'pending' AND scheduled_for <= now()
ORDER BY priority DESC, created_at ASC
LIMIT 50
FOR UPDATE SKIP LOCKED;

---

2. Rate-Pacing and Exponential Backoff

Rate Pacing: The Edge Function worker paces delivery (e.g., 5 to 10 emails per second), avoiding university firewall thresholds. Exponential Backoff: If an email encounters a temporary SMTP error (e.g., recipient inbox full or temporary server timeout), the status transitions to retry with an exponential delay: $$\text{delay} = 2^{\text{attempts}} \times 60 \text{ seconds}$$ Dead-Letter Audit*: Failed rows are moved to a dead-letter state after 5 retries, alerting the organizer to misspelled email addresses without blocking the rest of the queue.

This durable architecture ensures a 99.8% inbox delivery rate across all university email domains.