Does Your App Need a Queue? Three Symptoms
Does your app need a queue? Three symptoms say yes, and most apps show none of them. Here is how to tell, and the cheapest fix short of a queue.
Does your app need a queue? Probably not. Adding one buys you a new moving part, a new failure mode and a new thing to monitor, in exchange for solving a problem many apps do not have.
There are three symptoms that say otherwise. If none of them describes what you have built, close this and go build something else. If one does, the fix is usually smaller than a queue.
Symptom 1: a request is doing work the user is not waiting for
Someone uploads a file. Your handler saves it, generates three image sizes, runs it past a model, writes four database rows, sends an email, and then returns a response.
The user cared about one of those things: that the upload worked. Everything else happened while they watched a spinner.
The test: for each step in your slowest handler, ask whether the user's next action depends on it finishing. The steps that fail that test are queue candidates, and the email is almost always one of them.
Symptom 2: something times out and you raised the timeout
A platform returns a 504 after 30 seconds. You raised it to 60. Then 120. Now it works, mostly, until a larger input arrives.
Raising a timeout does not make work faster, it makes the failure rarer and worse. The 120-second version fails later, after more work has been done, with more partial state left behind, and now a user has been staring at a loading indicator for two minutes before being told it did not work.
The test: have you increased a timeout more than once for the same endpoint? If yes, that is the symptom. The work does not belong in a request.
This one bites AI apps hardest, because a model call that usually takes four seconds occasionally takes forty, and an agent doing several calls in sequence is firmly outside what a request should contain. Streaming the response solves the perception problem for a single call. It does not solve a multi-step task.
Symptom 3: a failure means redoing everything
Step four of six fails. There is no record of steps one to three. The only recovery is to start again, and starting again re-does work that already succeeded, which for anything billed per token means paying twice.
The test: when a long operation fails halfway, can you resume, or only restart? If only restart, you have no durable record of progress, and that is what a queue gives you more than anything else.
This is the symptom people underrate. They think of queues as a performance tool. The larger benefit is that work becomes a row somewhere, with a state, that survives a crash.
The ladder: three rungs, start at the bottom
Do not jump to a managed queue because an article mentioned one. There are three rungs and most apps should stop on the first or second.
Rung | What it is | Good for | Costs you |
|---|---|---|---|
1. A table | A `jobs` table with status and a worker polling it | Tens of jobs an hour | Almost nothing |
2. Built-in background tasks | Your platform's own background or scheduled function | Simple fire-and-forget | A platform dependency |
3. A real queue | Managed queue plus workers | Thousands of jobs, retries, priorities | A service to run and watch |
Rung 1 in practice
A table with `id`, `type`, `payload`, `status`, `attempts`, `last_error`, `created_at` and `updated_at`. A worker that selects the oldest pending row, marks it running, does the work, marks it done or failed. Run the worker on a schedule.
That is a queue. It is not a sophisticated one, it will not handle thousands of jobs a minute, and it handles the three symptoms above completely. For a small app, it is usually the right answer and it is boring in the way infrastructure should be.
The two details worth getting right from the start: lock the row when you claim it so two workers cannot take the same job, and cap `attempts` so a permanently failing job stops retrying rather than running forever. A retry loop against a paid API is an expensive way to learn this.
When you genuinely need rung 3
Move up when you need things a table makes painful: priority ordering, delayed scheduling, fan-out to many workers, dead-letter handling, or throughput where polling a table becomes the bottleneck.
Those are real needs and they arrive later than people expect. If you are not sure which rung you are on, you are on rung one.
What this does for an AI app specifically
Three things, all of which matter more than the latency win:
**Cost control.** Durable progress means a failure costs you the remaining steps, not all of them.
**Honest status.** A job row lets you tell a user "step 3 of 6" instead of showing a spinner and hoping.
**Outage survival.** When a provider has a bad hour, queued work waits and retries rather than failing in front of a user. What to do when your provider is down is a far shorter list if the work is already durable.
If you are still deciding what the work should even be, do you need an LLM for this task is the question above this one, and why your app is slow covers the diagnostic order if the symptom is latency rather than durability. The guide to building apps with AI covers where this sits in the wider build.
FAQ
Is a cron job a queue?
No. Cron runs on a schedule regardless of whether there is work. A queue runs because there is work. Cron is fine for genuinely periodic tasks and poor for work triggered by users.
Can I just use a background thread?
For a few seconds of non-critical work, sometimes. It dies with the process, which means a deploy or a crash silently loses the job. That is acceptable for a cache warm and not for anything a user paid for.
How do I know when rung 1 is no longer enough?
When the polling interval becomes a latency complaint, or when you start writing code to handle priorities and delays. Both are signals you are rebuilding a queue badly.
Sources: AWS documents the durability and retry semantics that distinguish a real queue from a loop in its Amazon SQS developer guide.
How did this land?
About the author
Developer Advocate
Steve builds something with Swarmz every week and writes up what worked, what broke, and what he'd do differently. Tutorials and hands-on guides are his lane.
