DEV Community

Cover image for Queue Mode in n8n Is an Availability Decision, Not a Capacity One
אחיה כהן
אחיה כהן

Posted on Edited on

Queue Mode in n8n Is an Availability Decision, Not a Capacity One

I run two production n8n instances. Both sit on identical 2 vCPU / 4 GB VPSes. One runs queue mode with a dedicated worker, Redis, and Postgres. The other runs plain single-process mode.

Here's the part that should change how you think about queue mode: on the busy one, even at the single busiest moment of a normal week, the number of executions in flight at once stayed small. My worker runs with --concurrency=5. The box never broke a sweat.

Queue mode is the most recommended "scaling" step in every n8n thread, and most people who follow that advice are buying RAM and operational complexity they will never use. Two production incidents show where the line actually is.

What the busy instance actually does

Most runs finish in well under a second. This is what real automation traffic looks like: webhooks arrive, a filter node rejects most of them, done. It is not a compute workload. It's a plumbing workload.

The price of admission is RAM: the worker is a whole second n8n process. Redis is a rounding error. You pay that before you've gained anything.

What queue mode actually bought me

Not speed. A sub-second execution is sub-second in either mode. The queue adds a Redis hop; nobody notices it, but nobody gains from it either.

What it actually bought me, in order of how often it mattered:

1. Crash isolation. A few AI workflows hung on a model call for hours before dying with an error. In regular mode, runs like that live inside the same process that serves your editor and your webhooks, and a memory-hungry one takes everything down with it. In queue mode each of them burned one worker slot out of five while everything else kept flowing. This is the real feature. It's an availability feature, not a performance feature.

2. Restart behavior. I can restart the worker (after an update, after a leak) without dropping incoming webhooks. The main process keeps accepting; the queue holds jobs until the worker returns.

3. A scaling path I've never used. If I ever need a second worker, it's one compose line. My peak in-flight count says that day is far away. That count also includes parent workflows sitting idle waiting on sub-workflow calls, so the true CPU-busy number is lower still.

What queue mode charged me

The migration race. Every version upgrade, docker compose up -d starts main and worker together, and both immediately try to run database migrations on the same Postgres. n8n has no cross-instance migration lock. The main wins; the worker crashes mid-migration with a MigrationExecutor.executePendingMigrations stack trace and sits there unhealthy. First time it happened (2.30.6 → 2.32.6, during a routine fleet upgrade), the worker sat unhealthy while I worked out whether my database was half-migrated.

It wasn't. The fix is boring and permanent: wait for main to report healthy, then docker restart the worker. It rejoins cleanly because the migrations are already done. But nobody tells you this before you switch, and an unhealthy worker after an upgrade looks exactly like a disaster until you know it's choreography.

A second thing to monitor. A dead worker in queue mode fails quietly: webhooks still return 200, jobs still enqueue, and nothing executes. In regular mode, when n8n is down, everything visibly fails, which is ugly but honest. Queue mode converts loud failures into silent backlogs. You need a health check on the worker specifically, not just on the UI.

The incident that queue mode couldn't touch

The one time this instance actually hurt, no execution mode would have saved it.

One gateway workflow, a filter in front of the bot on my own WhatsApp line, was receiving every event on the line and rejecting most of them in its first node. 11,880 executions per day, 96% finishing under a second. Pure no-op traffic.

The damage wasn't CPU. The box was idle. The damage was writes: n8n persists every execution by default, and my pruning cap (EXECUTIONS_DATA_PRUNE_MAX_COUNT=50000) quietly became the effective retention window: a couple of days. Several of my active workflows had zero saved executions left: a workflow failed three days earlier and there was no evidence it ever ran. Meanwhile execution_data grew to most of the database, and the nightly dumps bloated the backup directory.

The fix was two settings on one workflow: saveDataSuccessExecution: none, saveDataErrorExecution: all. The gate's no-op runs stopped being stored. Same traffic, far fewer writes.

Queue mode has no opinion about any of this. Workers don't reduce writes; they just move where the writing happens. If your n8n feels heavy, check what it's storing before you scale what it's computing.

Where the line actually is

My rule after running both modes side by side:

Stay in regular mode while all of these are true: your runs are mostly short (check with the query below), a single stuck execution taking the UI down for a minute is survivable, and you're on one box anyway.

Switch to queue mode when any of these arrives: individual runs that go multi-minute or memory-heavy (a hung AI call that runs for hours is the poster child), webhooks that must stay up while you restart things, or genuine horizontal scaling.

Notice what's not on the list: raw execution count. A busy week fits through --concurrency=5 because almost all of it is sub-second. Volume is the wrong trigger. Duration and blast radius are the right ones.

Two queries to run against your own instance before you add containers:

-- What fraction of your runs are sub-second no-ops?
SELECT (EXTRACT(EPOCH FROM ("stoppedAt" - "startedAt")) < 1)::int AS under_1s,
       count(*)
FROM execution_entity
WHERE "stoppedAt" IS NOT NULL
GROUP BY 1;

-- Your true peak concurrency (sweep line over start/stop events)
WITH ev AS (
  SELECT "startedAt" AS t,  1 AS d FROM execution_entity
   WHERE "startedAt" IS NOT NULL AND "stoppedAt" IS NOT NULL
  UNION ALL
  SELECT "stoppedAt", -1 FROM execution_entity
   WHERE "startedAt" IS NOT NULL AND "stoppedAt" IS NOT NULL
)
SELECT max(c) FROM (SELECT sum(d) OVER (ORDER BY t, d DESC) AS c FROM ev) x;
Enter fullscreen mode Exit fullscreen mode

If the first query says most of your load is sub-second, fix your filters before your architecture. If the second says your peak is under 10, queue mode is an availability decision, not a capacity one — make it for the crash isolation or don't make it at all.

These two instances run client-facing automation at my automation studio, so the "survivable downtime" bar is set by paying customers, not by my patience. That's why the busy instance got queue mode long before it needed a second worker.

What's your peak? Run the sweep query on your own instance and post the number. I am collecting evidence for a follow-up on how oversized most n8n deployments are. If you've crossed your worker concurrency in production, I especially want to hear what the workload was.

Top comments (0)