Start here: does the caller wait?#
TL;DRthe 30-second version
- A request triggers work. Run it synchronously (the caller blocks until it finishes and gets the result or error inline) or asynchronously (you accept it, put the work on a queue, return an acknowledgement now, and a worker does it later).
- Synchronous is simple: one call, one answer. But the caller's latency is the sum of every downstream hop, and if one dependency is slow or down, that failure travels straight back to the user.
- Asynchronous decouples the caller from the work. A queue absorbs traffic spikes, isolates a downstream outage, and retries failures — at the cost of no immediate result (you need polling, a callback, or a webhook), eventual consistency, and handling retries, idempotency, ordering, and messages that keep failing.
- The deciding question is never 'which is faster' — it's whether the caller needs the result to keep going. If yes, stay synchronous. If the work is slow, spiky, or best-effort and the caller can move on, add a queue.
Picture a request arriving at your service. To answer it, the service has to do some work: charge a card, resize an image, send a confirmation email, update a search index. The first decision is whether the caller has to wait for that work to finish. That single choice splits every design into one of two shapes.
The first shape is synchronous request/response. The caller sends the request and blocks — it holds the connection open and waits. The service does the work, then returns the result, or an error if something failed. The caller learns the outcome the moment the call returns. This is how a plain function call works, and how most HTTP APIs work by default: you ask, you wait, you get an answer.
The second shape is asynchronous. The service accepts the request but does not do the work inline. It writes the work down as a message and puts it on a queue — a durable buffer that holds work until someone is ready to process it. Then it immediately returns an acknowledgement: not the result, just a promise that the request was received and will be handled. Often this is an HTTP 202 Accepted, which means exactly that — accepted, not yet done. Later, a separate process called a worker pulls the message off the queue and does the actual work.
How each shape actually runs#
Synchronous is the one you already know. The caller sends the request. The service, still holding the connection open, calls whatever it needs — a database, a payment provider, another service — and each of those calls blocks too, in a chain. Only when the last one returns does the service build a response and send it back. The caller was parked the whole time. If any link in the chain throws an error or times out, that error propagates back up and the caller sees it directly. Simple to reason about: one thread of control, one answer.
Asynchronous splits that one thread into two halves that never meet. The first half is the producer: the service that received the request. It validates the request just enough to accept it, writes a message describing the work — say 'transcode video 8f3a into 720p' — and appends it to the queue. Then it returns the acknowledgement and closes the connection, all in milliseconds. The second half is the consumer, or worker: a separate process, often a whole pool of them, that continuously reads messages off the queue and does the real work. The producer and the worker run independently, at their own speeds, and only ever talk through the queue between them.
The costs a queue makes you take on#
A queue is not free. The moment you decouple the caller from the work, you inherit a set of problems that simply don't exist when the caller just waits. These are the ones an interviewer will push on, so know them by name.
- No immediate result. The caller got an acknowledgement, not an answer. You have to build a way to deliver the outcome later — polling, a callback, or a webhook — and that's extra machinery on both sides.
- Eventual consistency. There's now a window where the request is accepted but the work isn't done. A user who uploads a photo and refreshes might not see it yet. The system is correct, just not instantly, and you have to design the UI around that gap.
- Retries and at-least-once delivery. Queues retry a message if a worker crashes mid-way, which is what makes them reliable. But retrying means the same message can be delivered more than once. 'At-least-once' delivery — the common default — means duplicates are expected, not a bug.
- Idempotency. Because duplicates happen, the work must be safe to run twice — charging a card twice for one order is a disaster. You make it idempotent (running it again has the same effect as once), usually by attaching a unique id to each message and skipping any id you've already processed.
- Ordering. Two messages can reach a worker out of the order they were sent, especially with a pool of workers pulling in parallel. If 'set balance to 100' then 'set balance to 200' get reordered, you get the wrong answer. Preserving order costs you — usually parallelism.
- Dead-letter and poison messages. A message that fails every time — bad data, a bug — would otherwise retry forever and block the queue. After a few tries you route it to a dead-letter queue, a side channel for messages that can't be processed, so a human can look while the rest keep flowing.
There's also a subtler cost: async has a latency floor for small jobs. If the work takes 5 milliseconds, doing it synchronously answers the caller in about 5 milliseconds. Doing it through a queue means writing the message, waiting for a worker to poll it, running it, then delivering the result back — easily slower end-to-end than just doing the tiny job inline. A queue pays off when the work is heavy or bursty enough that decoupling is worth the overhead. For cheap, fast work the caller needs answered now, the queue is pure tax.
PredictA checkout endpoint runs three steps inline: charge the card (300 ms), send a receipt email (via a provider that is currently slow, 4 s), and update the loyalty-points balance (100 ms). Users are complaining checkout takes over four seconds. What do you move, and what does the caller's latency become?
Hint: Add up the hops for the sync number. Then split the steps by which ones the caller truly needs before it can move on.
Synchronous latency is the sum of every hop: 300 + 4000 + 100 = 4400 ms, and the whole thing is hostage to the one slow dependency (the email provider). Ask which steps the caller actually needs before it can continue. Charging the card, it does — if the charge fails, checkout must fail, so that stays synchronous. The receipt email and the loyalty update, it does not — the user doesn't need to wait for an email or a points tally before seeing 'order confirmed.' Move both onto a queue. Now the endpoint does the 300 ms charge inline, enqueues two messages (a couple of milliseconds), and returns. Latency drops to roughly 300 ms, and the slow email provider can take four seconds, or briefly fall over, without the user feeling it — the message waits in the queue and a worker retries. The one-line answer: keep the step whose result the caller needs synchronous; push the slow, best-effort steps to a queue.
| Synchronous | Asynchronous (queue) | |
|---|---|---|
| Caller gets | The result or error, inline | An acknowledgement (202); result comes later |
| Latency | Sum of every downstream hop | Fast ack; work finishes on the worker's own clock |
| A slow/failed dependency | Becomes the caller's latency and error | Absorbed by the queue; caller unaffected |
| Traffic spike | Overloads the caller and everything downstream | Buffered in the queue, drained at a steady rate |
| Consistency | Immediate — done when the call returns | Eventual — a window where it's accepted but not done |
| New problems you own | Few — one thread, one answer | Retries, duplicates, idempotency, ordering, dead-letters |
When a queue earns its keep#
Add a queue when the work has at least one of these shapes — and the more of them at once, the stronger the case:
- Slow work. Video transcoding, PDF generation, a report over millions of rows. Making a user hold a connection open for 30 seconds is a bad experience and ties up server resources; accept it, return 202, and finish it in the background.
- Spiky work. A flash sale sends ten times the normal traffic in a minute. A queue absorbs the burst and lets the workers drain it at a steady, safe rate — this is load leveling, and it keeps the spike from knocking the database over.
- Best-effort work. A notification, a cache warm, an analytics update. If it's a few seconds late, or retried, nobody cares. The caller has no reason to wait, so don't make it.
- Retryable work. Anything that talks to a flaky external service. A queue retries for you and holds the message through a downstream outage, so a temporary failure becomes a delay, not a lost request.
- Fan-out work. One event triggers many independent actions — an order placed notifies the warehouse, the email service, the analytics pipeline, and the recommendation engine. A queue lets one message fan out to many consumers without the producer waiting for any of them.
And do not add a queue when the caller genuinely needs the result to proceed. If a user submits a login and you must tell them right now whether it worked, that's synchronous — there is no result to defer. If a step's failure must fail the whole request (the card charge above), it stays inline. And if the work is fast and the traffic is calm, a queue only adds a latency floor and operational complexity — retries, idempotency, dead-letter handling, more to monitor — for no real benefit. Reaching for a queue reflexively, because it feels more scalable, is a common and expensive mistake. The queue should answer a specific pain: slow, spiky, best-effort, or must-survive-an-outage. No pain, no queue.
The decision, in one table#
Walk down these questions in order. The first one that clearly answers 'yes' usually settles it.
| Ask… | If yes → | Because |
|---|---|---|
| Does the caller need the result before it can continue? | Synchronous | There's nothing to defer — the answer is the point of the call |
| Is the work slow (seconds+), and the caller shouldn't hold a connection? | Async / queue | Return 202 now; finish in the background off the request path |
| Is the load spiky, and could a burst overload what's downstream? | Async / queue | The queue buffers the burst and levels it into a steady drain |
| Must the work survive a brief downstream outage? | Async / queue | A durable queue holds and retries the message until it succeeds |
| Is the work best-effort or fan-out (email, notify, index, analytics)? | Async / queue | The caller has no reason to wait; one message can feed many consumers |
| Is the work fast, calm, and the caller waiting on it? | Synchronous | A queue would only add a latency floor and operational complexity |
A common and healthy design is a hybrid: do the small, must-answer-now part synchronously, and enqueue the rest. The checkout charges the card inline (the caller needs that answer) and queues the receipt email and the loyalty update (it doesn't). You're not picking one shape for the whole system — you're picking, step by step, which work the caller must wait for and which it can hand off. That per-step judgement is the real skill, and exactly what the table above is for.
In the wild
- Payments and checkout: the card authorization is synchronous — you must tell the buyer immediately whether the payment went through — but the receipt email, fraud scoring, and ledger updates are commonly queued and processed just after.
- Media pipelines: YouTube, Instagram, and every image host accept an upload synchronously, return fast, and transcode or generate thumbnails asynchronously through a queue of workers. The 'still processing' state you see is eventual consistency made visible.
- Notifications and email: services like SendGrid or Amazon SNS/SQS exist precisely to make sending best-effort and retryable. Your app enqueues 'send this' and moves on; the delivery service handles retries and backoff behind the queue.
- Order fan-out: an 'order placed' event drops one message that fans out to inventory, shipping, email, and analytics consumers — each independent, each retried on its own, none of them blocking the customer's confirmation page.
- Reliable async — the transactional outbox: when a service must both update its database and publish a message, it writes the message into an 'outbox' table in the same database transaction, and a separate process relays those rows to the queue. That closes the gap where a crash could commit the data but lose the message. See the Transactional Outbox topic for the full pattern.
Pitfalls & gotchas
Isn't async just strictly better because the response is instant?
Faster to acknowledge, not faster to finish, and not free. The instant 202 hides the fact that the work still has to happen — and now you owe the caller a way to learn the outcome (polling, a webhook), plus idempotency, retries, ordering, and dead-letter handling. For fast work the caller needs answered now, a queue is slower end-to-end and much more complex. 'Instant' only means the acknowledgement is instant.
Why do I have to handle duplicates? I only sent the message once.
Because queues favor not losing messages over never repeating them. If a worker pulls a message, does the work, but crashes before confirming it's done, the queue can't tell whether the work happened — so it redelivers to be safe. That's at-least-once delivery, the common default, and it means the same message can be processed twice. You make the work idempotent (safe to run twice, usually via a unique message id you dedupe on) so a duplicate is harmless.
What happens when the workers can't keep up with the queue?
The queue grows — this is called lag or backlog, and it's the number to watch. A queue absorbs a short spike beautifully, but if the producers are permanently faster than the consumers, the backlog grows without bound and messages get old. The fixes are to add workers, speed up the work, or apply backpressure — signaling producers to slow down or shedding load — so the system degrades on purpose instead of falling over. See the Backpressure topic.
What's a poison message, and why does it need its own queue?
A poison message is one that fails every single time it's processed — corrupt data, a bug it trips, a schema it doesn't match. Without a safety valve it retries forever, wasting workers and potentially blocking everything behind it. After a few failed attempts you route it to a dead-letter queue, a side channel where failed messages wait for a human to inspect, while the rest of the traffic keeps flowing normally.
QuizA team moves their 'send welcome email' step off the signup request and onto a queue. Signup gets faster, but now some new users report getting two or three welcome emails. What's the most likely cause, and the right fix?
- The queue is broken and losing state — switch back to synchronous sending.
- At-least-once delivery is redelivering messages after worker retries; make the send idempotent by deduping on a unique message id.
- The workers are too slow — add more workers to stop the duplicates.
- Emails are inherently unreliable — there's nothing to do about it.
Show answer
At-least-once delivery is redelivering messages after worker retries; make the send idempotent by deduping on a unique message id. — Duplicate emails are the classic symptom of at-least-once delivery, the normal default for queues. When a worker sends the email but crashes or times out before confirming completion, the queue can't know the work finished, so it redelivers and a second worker sends the email again. This is expected, not a broken queue — so switching back to synchronous (choice A) throws away the latency win to fix a problem you can handle directly. Adding workers (choice C) doesn't help and can cause more duplicates. The fix is idempotency: attach a unique id to each send and record which ids you've delivered, so a redelivered message is recognized and skipped. Handling duplicates is the price of async reliability; idempotency is how you pay it.
In an interview
This is one of the most common design forks, and the interviewer is watching for whether you reach for a queue thoughtfully or reflexively. Don't declare async 'more scalable' and bolt a queue onto everything. Frame it as a trade, name the axis, and pick per step of work.
- Lead with the deciding question: does the caller need this result before it can continue? If yes, it's synchronous — there's nothing to defer. If no, a queue is on the table.
- Name the async wins concretely: spike absorption / load leveling, failure isolation, retries through a downstream outage, and fan-out. Tie each to the pain it solves, not just 'it scales.'
- Name the costs out loud before you're asked: no immediate result (polling/webhook), eventual consistency, and at-least-once delivery forcing idempotency, plus ordering and dead-letter handling. Volunteering these signals you've run one in production.
- Reach for the hybrid: keep the must-answer-now step synchronous and queue the rest, rather than forcing one shape on the whole flow. That's the senior move.
- Connect the neighbors: the queue itself (Message Queue), what to do when consumers fall behind (Backpressure), and how to publish a message reliably alongside a database write (Transactional Outbox). Knowing where async breaks — the outbox gap — is what separates a strong answer.
PredictAn interviewer says: 'Design a service that lets users request a large data export — it can take a few minutes to build. Sync or async?' What's the strong answer?
Hint: Slow work the caller shouldn't hold a connection for — but once you go async, how does the caller ever get the export, and how do you avoid building it twice?
Async, and say why in trade terms. The work is slow (minutes), so holding an HTTP connection open the whole time ties up a server thread, and any network blip loses the whole export. So accept the request, return 202 with an export id, and enqueue the job. Then address the cost you took on: the caller no longer gets the result inline, so give them a way to learn the outcome — poll a status endpoint with the export id ('pending' to 'ready'), or send a webhook or email with a download link when it's done. Make the job idempotent keyed on the export id so a redelivered message doesn't build the same export twice, and route a repeatedly-failing job to a dead-letter queue. What makes the answer strong isn't 'I used a queue' — it's that you named why sync fails here (slow work, fragile long connection), then owned the async costs instead of pretending they vanish.
References
- Designing Data-Intensive Applications, Ch. 11 (Stream Processing) — Kleppmann on message queues, asynchronous delivery, and the guarantees (at-least-once, ordering) that come with them.
- Enterprise Integration Patterns — Message Queue & Dead Letter Channel — The canonical catalog of async messaging patterns, including dead-letter channels and guaranteed delivery.
- AWS — What is a Message Queue? — A clear vendor-neutral primer on decoupling with queues, load leveling, and asynchronous processing.
- Microsoft — Asynchronous Request-Reply pattern — How to return 202 Accepted and deliver the eventual result via polling or a callback.
- microservices.io — Transactional Outbox — Publishing a message reliably alongside a database write — reliable async, and where naive async breaks.