Aug 24, 2026

18 min read

WebSocket Scaling: From One Server’s Limits To A Multi-Node Architecture

Following up on the previous post about how WebSocket establishes connections and secures them, say your realtime chat feature has been running smoothly: solid auth, wss:// everywhere, a few thousand users online every night. Then one day the product lands on the homepage of a major news site. Online users jump from 5,000 to 80,000 in an hour. The server starts rejecting new connections, memory climbs in a straight line, and when you scramble to spin up a second server behind the load balancer, something strange happens: user A sends a message that user B never receives — even though both are online.

The problem sits at the core of what WebSocket is: every connection is stateful (the server has to remember each connection’s state — which user, which channels they’re subscribed to) and long-lived (a connection stays open for minutes or hours instead of the tens of milliseconds a single HTTP request takes). With plain HTTP, a load balancer can throw any request at any server, because each request carries everything needed to handle it. With WebSocket, user B’s connection lives on exactly one specific server — and if user A’s message lands on a different server, it has no way to reach B.

This post walks through the two axes of scaling WebSocket: vertical scaling — how many connections a single server can hold and what stops it from holding more, and horizontal scaling — how multiple servers share the load, from load balancing and sticky sessions to a pub/sub backplane and autoscaling. Each section maps to AWS infrastructure so you can picture how it plays out in practice.

1. Why is WebSocket harder to scale than plain HTTP?

Before getting into solutions, it helps to understand why the scaling techniques you already know from HTTP don’t carry over directly.

An HTTP request is stateless: it carries everything (URL, headers, token, body) needed for any server to handle it, and once it’s done the connection either closes or returns to a pool. To scale, you just add servers and let the load balancer spread requests evenly — no server needs to know what any other server is doing.

WebSocket breaks both of those assumptions:

A connection is pinned to one server. After the handshake, the TCP connection between client and server stays open continuously. All of that connection’s state — the user’s identity, which rooms they’ve joined, any buffered outgoing messages — lives in that one server’s memory. No other server can see it.
A connection lives a long time. An HTTP request occupies server resources for tens of milliseconds. A WebSocket connection occupies resources continuously for as long as the user stays online — even when no message is being sent. 80,000 online users means 80,000 connections open at once, each one costing a file descriptor, memory, and a share of the event loop’s attention.

These two properties raise two questions, which map to the rest of the post:

How far can a single server go, and how do you push that ceiling higher? (vertical scaling)
Once you’re forced to run multiple servers, how do they coordinate — especially when a user on one server needs to receive a message from a user on another? (horizontal scaling)

2. Vertical Scaling: how many connections can one server hold?

Vertical scaling means increasing capacity by using a more powerful machine or tuning it to serve more, rather than adding machines. For WebSocket, the concrete question is: what determines a server’s maximum connection count?

2.1. What one connection costs

Every open WebSocket connection consumes resources at several layers:

File descriptor. On Linux, every TCP socket is a file descriptor (an integer the kernel uses to identify an open I/O resource — files, sockets, and pipes all count). Each connection takes exactly one fd, and every process has a limit on how many fds it can hold open at once.
Kernel buffers. The kernel allocates a send and receive buffer pair per socket, each ranging from a few KB to a few hundred KB depending on configuration. This is usually the biggest memory cost of an idle connection.
TLS session state. If you’re using wss:// (which you should be), each connection also holds encryption keys and buffers for TLS records.
Userland object. On the Node.js side, each connection is an object on the V8 heap: a ws library instance, its event listeners, message queues, plus whatever application state you attach yourself (user id, subscription list).

Add it all up and an idle connection costs roughly a few dozen KB. That sounds small, but multiplied by 100,000 connections it’s several GB of memory — before any real traffic even hits.

2.2. OS limits, and a common misconception

The first wall you’ll hit is the fd limit. On many Linux distros, a process can open only 1024 fds by default — meaning your server dies at around 1000 connections even with plenty of hardware to spare. This limit is raised with ulimit -n (per-process limit) and fs.file-max (system-wide limit), up into the hundreds of thousands or beyond.

A common misconception: “a machine only has 65535 ports, so it can only hold 65535 connections.” This is wrong on the server side. The 65535 limit applies to ephemeral ports (the temporary port range a machine uses as an outgoing source when it actively opens a connection) — that’s a client-side limit when connecting to the same destination. On the server side, every connection arrives on the same listening port (say, 443); the kernel tells connections apart using the four-tuple of source IP, source port, destination IP, and destination port. Since every client has a different source IP and port, a server can hold millions of connections on a single listening port — as long as it has enough fds and memory.

Beyond fds, a couple more kernel parameters are worth knowing once you go further: net.core.somaxconn (the length of the queue for connections waiting to be accepted) and the default kernel buffer sizes (net.ipv4.tcp_rmem, tcp_wmem) — shrinking buffer size lets you fit more idle connections into the same amount of RAM, at the cost of lower per-connection throughput.

2.3. Limits on the Node.js side

Once the OS limits are out of the way, the next ceiling is the runtime itself. Node.js handles I/O through a single-threaded event loop: one thread processes events (an incoming message, a closed socket, a timer) one at a time. This model is a great fit for lots of idle connections — 100,000 silent sockets cost almost no CPU. But it has two bottlenecks:

Broadcasting is O(N). Sending one message to 50,000 connections (a system-wide announcement, a message in a large room) means the event loop has to serialize and write to each socket in turn. While that’s happening, everything else — including accepting new connections and answering pings — has to wait in line.
Heap and GC. Hundreds of thousands of long-lived connection objects on the heap make every garbage collection cycle (V8’s process of scanning and reclaiming memory that’s no longer in use) take longer, causing pauses that show up as message latency spikes.

You can measure your own per-connection memory cost to find out where your ceiling actually is. The following script opens connections to a server in batches and logs memory usage:


import WebSocket from 'ws'
 
const sockets: WebSocket[] = []
 
async function openBatch(count: number) {
  for (let i = 0; i < count; i++) {
    const ws = new WebSocket('wss://localhost:8443', { rejectUnauthorized: false })
 
    ws.on('error', () => {})
 
    sockets.push(ws)
  }
 
  await new Promise((resolve) => setTimeout(resolve, 2_000))
}
 
async function main() {
  for (let opened = 0; opened < 10_000; opened += 1_000) {
    await openBatch(1_000)
 
    const { heapUsed, rss } = process.memoryUsage()
 
    console.log(`${sockets.length} sockets | heap ${(heapUsed / 1e6).toFixed(1)}MB | rss ${(rss / 1e6).toFixed(1)}MB`)
  }
}
 
main()

Run this on the client side, and on the server side track process.memoryUsage() the same way — dividing the memory delta by the connection count gives you the average per-connection cost for your own application, which is far more accurate than any benchmark number you’ll find online.

2.4. The ceiling on vertical scaling

With good tuning, a Node.js server can comfortably hold hundreds of thousands of connections. So why not just keep buying bigger machines? Three reasons:

Cost grows non-linearly. A 128GB machine costs far more than four 32GB machines, and the gap widens the further up you go.
A physical ceiling. Eventually there’s no bigger machine to buy — or a single-threaded event loop can’t keep up with the message volume even with RAM to spare.
A single point of failure. This is the most important reason. A server holding 500,000 connections means that when it dies — a bad deploy, a kernel panic, hardware failure — 500,000 users drop at once, then reconnect at once. That reconnect storm can take down whatever’s left of the system.

Vertical scaling is therefore a first optimization step, not the destination: push the limits so each node can carry more, and you’ll still end up running multiple nodes.

Mapping to AWS: vertical scaling means picking a larger instance size (memory-optimized instances like the r family usually fit WebSocket well, since memory is the resource that runs out first). Note that each instance type also has its own network bandwidth cap and its own limit on tracked connections in the security group (conntrack) — a small instance may bottleneck on network before it ever bottlenecks on RAM.

3. Horizontal Scaling: running multiple nodes

Horizontal scaling means adding more servers to share the load instead of scaling up a single one. With stateless HTTP, this is nearly free. With WebSocket, it raises three new problems: which node a connection lands on (load balancing and sticky sessions), how a message travels across nodes (fan-out), and how nodes get added or removed safely (autoscaling and graceful drain).

3.1. Load balancing for WebSocket

A load balancer (LB) sits between clients and the server fleet, accepting every incoming connection and distributing it to a node behind it. For WebSocket, an LB behaves differently from serving plain HTTP in two ways:

The LB has to hold connections open for a long time. After the handshake, the LB stops being a request-splitter and becomes a pipe: every frame between client and node flows through it for the connection’s entire lifetime. That means the LB burns fds and memory per connection just like the server does — it has to scale right alongside the fleet.
Idle timeout is the classic trap. Most LBs automatically close a connection that’s carried no data for some period (often 60 seconds by default). A user with a chat tab open but not typing for a few minutes gets silently disconnected. The fix is a heartbeat: the server sends a periodic ping frame (say, every 30 seconds) and the client answers with a pong — this keeps the connection alive through the LB and lets the server detect connections that have quietly died.

At the protocol layer, there are two choices. A layer 4 LB operates at the TCP level: it just forwards bytes without understanding the content, so it’s fast and nearly transparent to WebSocket. A layer 7 LB understands HTTP: it can read the handshake, route by path or header, and terminate TLS on the server’s behalf — but it has to explicitly support the Upgrade and adds a bit of latency. Both work fine for WebSocket; pick layer 7 when you need smart routing and TLS offload, layer 4 when you need maximum throughput and to preserve TCP-level information.

Mapping to AWS: both ALB (layer 7) and NLB (layer 4) support WebSocket. ALB handles the Upgrade natively, terminates TLS, and has a configurable idle timeout (raise it, and still keep the heartbeat); NLB forwards TCP directly with lower latency. I compared the two in detail in ALB or NLB and ALB under the hood.

3.2. Sticky sessions: how much do you actually need?

Sticky sessions (also called session affinity) is the LB mechanism that guarantees requests from the same client always land on the same node, instead of being spread randomly.

It might sound like WebSocket always requires sticky sessions — but there are actually two different situations to tell apart:

Plain WebSocket doesn’t need stickiness for the connection itself. A connection is a single TCP stream; the LB picks a node exactly once, at handshake time, and every frame after that automatically follows that same connection. There’s no “second request” that could wander off to a different node.
Stickiness becomes mandatory when the handshake spans multiple requests. The classic case is Socket.IO in its default mode: it starts with several HTTP long-polling requests (the client repeatedly sends HTTP requests asking for new data, simulating realtime behavior over plain HTTP) before upgrading to WebSocket. Those polling requests all have to land on the same node — any other node has no idea who this session belongs to. Without stickiness, the client gets random “session not found” errors.

Sticky sessions have a side effect that’s easy to miss: load skews over time. Because connections live a long time and are never redistributed, a node that’s been running longer accumulates connections, while a freshly added node stays nearly empty — the LB can only hand it new connections. After a rolling deploy, you might find the first node to restart is carrying twice the load of the last one. The fix is to actively rebalance: the server periodically asks a portion of an overloaded node’s clients to reconnect (the client automatically comes back through the LB and lands on a less-loaded node), or you accept the skew and scale to the heaviest node.

Mapping to AWS: ALB supports sticky sessions via a cookie at the target group level — turn it on if you’re using Socket.IO with polling fallback. Details on target groups and sticky sessions are in Elastic Load Balancer overview.

3.3. Cross-node fan-out: the core problem

This is exactly what stopped user A’s message from reaching user B at the start of this post. User A is connected to node 1, user B to node 2. Node 1 receives A’s message, checks the connections it holds in memory — B isn’t there. As far as node 1 is concerned, user B simply doesn’t exist.

The standard fix is adding a backplane (a shared communication channel every node connects to, used to pass messages between nodes) running a pub/sub model (publish/subscribe — a sender publishes a message to a named channel, and every party subscribed to that channel receives it, with neither side needing to know the other). Redis Pub/Sub is the most common choice for this role because of its low latency and because most systems already have Redis running somewhere.

Here’s the path a message takes:

User A sends a message to room 42 over their connection to node 1.
Node 1 doesn’t try to find the recipient itself — it publishes the message to the room:42 channel on Redis.
Redis pushes the message to every node subscribed to room:42 — including node 2, since node 2 holds user B’s connection (a member of room 42).
Node 2 receives the message from Redis, checks its own local connection list, and writes the message to user B’s socket.

The key point: each node only subscribes to the channels its own users care about, and is only responsible for writing to its own local sockets. No node ever needs to know which node a given user is on — Redis handles the routing.

A minimal implementation with ws and ioredis (a Redis client for Node.js; you need two separate Redis connections, since a connection that’s entered subscribe mode can no longer be used to publish):


import { WebSocketServer, WebSocket } from 'ws'
import Redis from 'ioredis'
 
const pub = new Redis()
const sub = new Redis()
const wss = new WebSocketServer({ port: 8080 })
 
const roomMembers = new Map<string, Set<WebSocket>>()
 
sub.on('message', (channel, payload) => {
  const members = roomMembers.get(channel)
 
  if (!members) {
    return
  }
 
  for (const socket of members) {
    if (socket.readyState === WebSocket.OPEN) {
      socket.send(payload)
    }
  }
})
 
wss.on('connection', (socket) => {
  socket.on('message', async (raw) => {
    const { type, room, body } = JSON.parse(raw.toString())
 
    const channel = `room:${room}`
 
    if (type === 'join') {
      if (!roomMembers.has(channel)) {
        roomMembers.set(channel, new Set())
 
        await sub.subscribe(channel)
      }
 
      roomMembers.get(channel).add(socket)
    }
 
    if (type === 'chat') {
      await pub.publish(channel, JSON.stringify({ room, body }))
    }
  })
})

Notice that the sending node also receives its own message back through Redis — that’s a feature, not a bug: every node handles messages through the same code path, even when the sender and recipient happen to share a node.

Two limitations of Redis Pub/Sub worth knowing before you rely on it:

It’s fire-and-forget. Redis doesn’t store messages: whichever node is subscribed at that moment gets it, and a node that’s mid-restart at exactly the wrong time loses the message permanently. For ordinary chat this is usually fine (clients typically re-fetch history from the database on reconnect), but if you need a guarantee that no message is lost between nodes, look at a message broker with storage, like Kafka — I covered this in detail in Kafka architecture deep dive. In exchange, Kafka has higher latency and is considerably heavier to operate, so don’t reach for it until you actually need it.
Redis itself becomes the next bottleneck. Every message between nodes passes through a single Redis instance. Once total message volume outgrows one instance, you have to shard channels across multiple Redis instances (e.g. hashing the room name to pick an instance) — at which point complexity increases substantially.

Mapping to AWS: ElastiCache for Redis is the natural choice for the backplane. Place ElastiCache in the same region and VPC as the WebSocket fleet to keep pub/sub latency under a millisecond.

3.4. Autoscaling: adding and removing nodes without dropping users

With an LB and a backplane in place, the last piece is autoscaling — automatically adding nodes when load rises and removing them when it falls. WebSocket differs from HTTP autoscaling in two major ways.

First, CPU is the wrong metric. A node holding 100,000 idle connections has CPU usage near zero — an autoscaler watching CPU will conclude the node is idle and scale it down, when in fact it’s already full. The right metric for WebSocket is open connection count and memory: the application counts its own connections and pushes that number to the monitoring system as the scaling signal.

Second, scaling in is riskier than scaling out. Adding a node is easy: it joins the LB and starts accepting new connections. But removing a node means cutting tens of thousands of live connections. Do it abruptly, and every one of those clients drops at once and reconnects at once — the same reconnect storm from section 2.4, except now you’re causing it yourself with every scale-in or deploy. The right process is called graceful drain:

Deregister the node from the LB first — it stops receiving new connections, but existing ones stay alive.
The node actively asks its own clients to reconnect: a close frame or an application-level message, spread out over time rather than sent all at once.
Clients reconnect through the LB and land on the remaining nodes. Clients should implement reconnect with jittered backoff — retrying after a growing delay plus a random jitter, so that even if a node dies suddenly, clients don’t all pile back in at the same instant.
Once the drain window ends (a few minutes, say), the node closes whatever connections remain and shuts down.

Deploying a new version is really this same process repeated node by node — so investing in graceful drain once pays off every day, not just during scale-in.

Mapping to AWS: an Auto Scaling Group manages the fleet, scaling on a custom metric (the connection count the application pushes to CloudWatch) instead of CPU. A target group’s deregistration delay gives a node time to drain before it’s fully removed, and a lifecycle hook lets you run drain logic before an instance is terminated — both concepts are covered in Auto Scaling Group overview.

4. Conclusion

Back to the story at the start: the server rejected connections because it hit the fd limit, and user A couldn’t message user B because the two were on separate nodes with no shared communication channel. Both are consequences of the same root cause: a WebSocket connection is stateful and long-lived, so it can’t scale the way stateless HTTP does. The key points to remember:

What one connection costs comes down to file descriptors, kernel buffers, and heap objects — adding up to a few dozen KB per idle connection. The 65535-port limit is a client-side limit, not a server one; a server bottlenecks on fd limits and memory first.
Vertical scaling has a ceiling: good tuning lets a Node.js node hold hundreds of thousands of connections, but a big machine is a single point of failure — one node dying triggers a massive reconnect storm.
A WebSocket load balancer has to hold connections open for a long time; remember to raise the idle timeout and add a heartbeat. Sticky sessions are only mandatory when the handshake spans multiple requests (like Socket.IO’s polling fallback), and they come with a side effect of gradually skewed load.
A pub/sub backplane is the heart of horizontal scaling: a node publishes messages to Redis instead of hunting for the recipient itself, and each node only writes to its own local sockets. Redis Pub/Sub is fire-and-forget — reach for Kafka if you need message durability.
Autoscale on connection count, not CPU, and invest in graceful drain: deregister from the LB, ask clients to reconnect gradually, then shut the node down. A daily deploy is really this same process.

And if you’d rather not operate all of this yourself, managed services like AWS API Gateway WebSocket handle the connection layer for you so you only write the logic — at the cost of per-message pricing and less control, a trade-off worth considering for a small team.