Put a hard limit on every queue before it fills your disk

The claim

Every queue in your system — the job queue, the message buffer, the retry backlog, the in-memory list of pending work — has a maximum size, and if you did not choose that maximum, it is whatever your memory or disk allows, discovered at the worst possible moment. An unbounded queue does not prevent overload; it delays and amplifies it, converting a period of high load into a resource exhaustion that takes down the whole system. A bounded queue that rejects work when full is a system that degrades; an unbounded one is a system that collapses.

Why unbounded queues fail worse than the load they absorb

A queue exists to smooth a mismatch between how fast work arrives and how fast it is processed. When arrival briefly exceeds processing, the queue holds the excess until the processor catches up — which is exactly what a queue is for, and it works well as long as the imbalance is temporary. The failure comes when the imbalance is not temporary: work keeps arriving faster than it is processed, the queue grows without limit, and now instead of a slowdown you have a queue consuming all available memory or disk until the process is killed or the disk fills and everything that needs to write fails at once.

The unbounded queue has taken a survivable condition — more work than you can process right now — and converted it into an unsurvivable one: resource exhaustion that fails the entire system, including all the work that had nothing to do with the overload. It did not absorb the problem; it stored it up and detonated it.

A bounded queue forces a decision at the right time

Setting a maximum size forces you to answer, in advance and calmly, the question the unbounded queue answers for you catastrophically: what should happen when more work arrives than can be handled? A bounded queue makes you decide the rejection behaviour deliberately:

queue = Queue(maxsize=10_000)   # not unlimited

try:
    queue.put(job, block=False)
except QueueFull:
    # a decision you get to make, instead of one made for you:
    reject_with_backpressure(job)   # tell the caller to slow down / retry later

When the queue is full, you reject the new work — and rejecting is a feature, not a failure. The caller learns immediately that the system is saturated and can respond sensibly: slow down, retry later, shed its own load. This is backpressure, and it is how a well-built system communicates "I am at capacity" instead of silently accepting work it cannot do until it dies trying.

Choose the rejection behaviour to fit the work

What to do with rejected work depends on what the work is, and the choice matters:

• Reject and signal the caller. For synchronous requests, return a "try again later" status so the caller backs off. This is the right default for user-facing work — a clear, immediate refusal beats an accepted request that never completes.
• Drop the oldest. For streams where only recent data matters — live metrics, position updates — discarding the oldest queued item to make room for the newest keeps the freshest data flowing and the stale data is no loss.
• Drop the lowest priority. When work has priority, a full queue should shed the least important item, not refuse the most important new one.
• Spill to durable storage. When no work may be lost, a full in-memory queue can overflow to disk or a durable broker — but that durable store also needs a bound, or you have only moved the unbounded problem somewhere slower.

The wrong choice is having no choice, which is what an unbounded queue gives you.

The queues you did not know were queues

The obvious message queue is easy to bound; the dangerous ones are the implicit queues you did not label as queues. A list of pending requests waiting on a slow downstream service is a queue. Unacknowledged connections waiting to be accepted are a queue. Buffered log lines waiting to be shipped are a queue. Each of these has a maximum size that is either chosen by you or imposed by resource exhaustion, and each can take down a system by growing without limit while nobody is watching it grow, because nobody thought of it as a queue that needed a bound.

The habit worth building is to look for every place work accumulates while waiting for something slower, recognise it as a queue, and ask what its bound is. If the answer is "however much memory we have", that is an unbounded queue wearing a disguise, and it will fail the same way the obvious one would.

Bound it, then watch how full it gets

A bounded queue should be monitored, because its depth is one of the best early-warning signals you have. A queue that is usually near-empty and is now consistently half-full is telling you that processing is falling behind arrival — the imbalance that, unbounded, would eventually exhaust your resources, caught early while there is still time to add processing capacity or investigate the slowdown. Alert on sustained depth, not just on the queue being full, because full is the emergency and rising toward full is the warning that lets you avoid it:

queue_depth / queue_maxsize > 0.7 for 5 minutes  ->  warn
queue rejecting work                             ->  page

What to do

Find every queue in your system, including the implicit ones, and confirm each has a maximum size you chose rather than one your hardware will choose for you under load. For each, decide what happens when it fills, matching the rejection behaviour to the nature of the work. Then monitor the depths so a growing backlog warns you before it becomes a rejection and a rejection warns you before it becomes an outage. A system built from bounded queues degrades gracefully under more load than it can handle; a system built from unbounded ones works perfectly right up until the moment it does not work at all.