Scaling past your MVP: adding Kafka and BullMQ without downtime
Your MVP is working. Users are signing up, the product is sticky, and the graphs are finally pointing up and to the right. Then the pager goes off: a burst of traffic hits an endpoint that does too much work in-line, requests pile up behind a slow third-party call, and the whole app feels slow for everyone. This is a good problem — it means people care — but it needs a different architecture than the one that got you here.
The instinct is to rewrite. Don’t. You can move to an event-driven system incrementally, one hot path at a time, with no maintenance window. Here’s the sequence we use.
1. Find the work that doesn’t need to be synchronous
Most latency problems come from doing work inside the request that the user doesn’t need to wait for: sending email, generating thumbnails, syncing to a CRM, calling an LLM. The first move isn’t Kafka — it’s a job queue. Pull that work out of the request and into a background worker with BullMQ, and your p95 drops immediately because the request returns as soon as the job is enqueued.
2. Make jobs safe to retry
Background work fails — networks blip, providers rate-limit. The rule that keeps queues honest is idempotency: a job should be safe to run twice. Key each side effect on a stable id so a retry is a no-op, not a duplicate charge or a second email. Pair that with sensible retry and backoff, and a dead-letter queue for the handful that never succeed.
- Idempotency keys on every external side effect
- Exponential backoff with a capped attempt count
- A dead-letter queue you actually monitor
- Rate limiters that respect your providers’ quotas
3. Reach for Kafka when many things care about one event
A queue is a great fit when one producer hands work to one kind of worker. But as the system grows, a single business event — order placed, user upgraded — often needs to fan out to billing, analytics, search indexing and notifications at once. That’s where Kafka earns its keep: producers publish events to a topic, and independent consumer groups each read at their own pace. Services stop calling each other directly and start reacting to a shared log of what happened.
A queue moves work. A log of events lets your system grow new features without touching the code that emits them.
4. Migrate one path at a time
Ship the queue first and measure. Introduce the event bus behind the scenes — publish events while the old synchronous code still runs, compare, then cut consumers over one by one. Because each step is additive and reversible, you never need a big-bang release or a downtime window. By the end, the monolith has quietly become a set of services coordinated by events, and you did it while shipping features the whole time.
That’s the whole trick: no rewrite, no heroics — just moving work off the request path, making it safe to retry, and letting events fan out as the system grows.