WebSockets in System Design: When You Need a Persistent Connection

This article is the first of the real-time communication pair; the reading order runs from here to the push-strategy comparison, and it arrives with a debt recorded in two earlier articles. The API gateway piece flagged the dilemma without solving it: gateways designed for request-response can hold worker threads hostage to idle connections, so long-lived traffic must either pass through or bypass, and which one to do depends on knowing what a websocket actually is. And the foundations put the problem in one sentence: in client server architecture, the server cannot start a cycle. This is the article about the connection that lets it.

A websocket is a single TCP connection, begun as an ordinary HTTP request and then upgraded, over which both sides can send framed messages at any time, no request needed to justify a response. The protocol is standardized in RFC 6455, and its reason for existing is latency: once the handshake is paid, a message costs one hop, not one cycle. That is why this pair’s prerequisites are what they are; latency is the currency the connection saves, and the reverse proxy is the infrastructure the handshake has to survive on its way in.

The honest opening is the same one the gateway piece used: most systems do not need this. If the client can ask, HTTP request-response is simpler, cacheable, and debuggable, and the HTTP-native ways of faking server push, long polling and server-sent events, cover the cases where the server talks more than it listens. Websockets earn their complexity only when the conversation is genuinely two-way and continuous: chat, collaborative editing, live games, trading, presence. The comparison of push strategies belongs to the next article; this one is about the machinery, what the connection is, how it opens and stays open, and what it costs to scale.

What follows defines the protocol, prices it against HTTP, walks the full lifecycle of one connection, and then takes the scaling question seriously, because the state that makes a websocket useful is the same state that makes a fleet of them hard. Along the way it pays three small debts: the gateway’s pass-through question, the balancer’s sticky problem, and the delivery guarantees websockets famously do not give.

What are websockets

Websockets are a transport protocol, standardized in RFC 6455, that turns a single TCP connection into a full-duplex channel: after an HTTP-based handshake, client and server exchange small framed messages in either direction, at any time, over the same connection. The connection (not the request) becomes the unit of conversation.

Three properties define the protocol. Full-duplex: both sides send whenever they like; neither waits to be asked. Framed: data travels in frames with opcodes (text, binary, and the control frames that keep the connection honest) so one TCP stream carries many messages without one devouring the others. Origin-checked: the handshake is an HTTP request, so browsers attach an Origin header and servers can refuse connections from pages they do not recognize; a security property that ad-hoc polling hacks never had.

The framing is worth one more look, because it is the quiet engineering. A websocket message can be split across frames; each frame carries its own length, so a receiver never parses an endless byte flood as one message. Control frames (ping, pong, close) ride the same stream without disturbing data. And every client-to-server frame is masked, a defense against intermediaries that might mistake websocket traffic for ordinary HTTP requests and poison a cache with it. None of this is visible to the application; all of it is why the connection survives real networks.

Two optional layers ride on top. The handshake can negotiate a subprotocol (the Sec-WebSocket-Protocol header) so client and server agree on what the frames mean: JSON chat messages, binary game state, signaling for a call. And the connection can negotiate compression (permessage-deflate), trading CPU for bandwidth on verbose text. Neither is required; both are worth knowing, because “the frames” is an empty phrase until something defines their contents, and that definition is the application’s own contract to write.

Websocket vs http

The comparison is not a rivalry, the two solve different shapes of conversation. HTTP is half-duplex by design: the client asks, the server answers, and between answers the server is mute; a rule that lives in the request response cycle itself. Websockets are full-duplex: after one handshake, both sides send when they have something to say. The cost asymmetry matches the shape: HTTP pays per-request overhead (headers, connection setup or pool checkout, the whole cycle) while a websocket pays once and then sends frames whose headers begin with two bytes instead of hundreds.

Where the difference bites is server initiative. Under HTTP, the only way for the server to say something unprompted is to wait for the client to ask again (polling) or to hold the client’s question open until an answer exists, long polling. Both smuggle push through the request cycle, and both have honest uses; the next article compares them properly. What neither delivers is a message the instant it matters, while the client is idle and the connection quiet. That instant is the websocket’s whole reason to exist: one hop, no cycle, no asking.

The trade runs the other way too, and it should be said plainly: an upgraded connection is a stateful one. It pins memory on one server for its whole life; it cannot be load-balanced per request; it forfeits the per-request indirection that makes HTTP fleets replaceable. HTTP connections are cheap to abandon and cheap to rebalance; websocket connections are neither. Choosing websockets is choosing to hold state at the connection layer, which is why the scaling section below is the longest in this article.

One more asymmetry, quieter than the rest: everything the HTTP ecosystem built assumes cycles. Proxies cache responses; websockets are uncacheable by construction. Access logs, metrics, and tracing tools key on requests; a websocket is one long request that never ends, so its traffic needs custom instrumentation to even be visible. Retry and idempotency machinery works per request; on a socket, redelivery is the application’s own problem. None of this makes websockets wrong; it makes them a different operating regime, with the observability bill paid separately.

Websocket connection lifecycle

One connection, six numbered steps from stranger to closed, and each step is a place where real deployments fail.

1. Open. The client sends an ordinary HTTP GET carrying two special headers, Upgrade: websocket and Connection: Upgrade, plus a random Sec-WebSocket-Key.
2. Agree. A willing server answers 101 Switching Protocols with a Sec-WebSocket-Accept value derived from the key: proof it speaks RFC 6455, not just HTTP with decoration.
3. Upgrade. From this byte onward the TCP connection is no longer HTTP: both sides switch to sending websocket frames, and every HTTP-speaking hop in between must have passed the Upgrade headers through untouched.
4. Exchange. Data flows in both directions as frames (text or binary, whole or split) while the application sends and receives as if the network were a pipe that talks back.
5. Stay proven. Either side may send a ping; the other must answer with a pong; the protocol’s own dead-peer detection, because TCP alone will happily report a dead connection as fine.
6. Close. The shutdown is a handshake too: a close frame from one side, its acknowledgment from the other, then TCP teardown, so neither side mistakes a deliberate end for a network failure.

Step 3 is where the reverse proxy earns its keep. The Upgrade headers are hop-by-hop; intermediaries are entitled to strip them, and some older ones do, so a websocket deployment is really a test of its infrastructure chain: every hop must forward the upgrade, none may buffer the stream (buffering a websocket is philosophically wrong and operationally fatal), and idle timeouts must be tuned for connections measured in hours, not seconds. Handshakes that worked in development die in staging for infrastructure reasons more often than for code reasons, which is why the lifecycle is worth knowing cold.

The handshake’s price is worth counting once: TCP connect, TLS, the upgrade round trip; one to two round trips before the first frame, where an ordinary HTTP request would already be finished. That cost is why websockets only make sense amortized: the connection pays it once and then sends messages for hours. A client that connects and disconnects constantly is paying the most expensive possible toll for its traffic; a pattern that makes tail latency worse, not better.

Scaling websocket connections

Scaling HTTP taught one lesson: stateless servers can be balanced per request, so fleets grow by adding machines. Websockets delete that lesson. A websocket’s state (the connection, its subscriptions, its half-finished messages) lives in the memory of exactly one server, so scaling is no longer per-request; it is per-connection. The first consequence is routing: the balancer’s usual freedom to send request eleven to a different server than request ten is gone. Either the balancer pins the connection to the server that holds it (sticky routing, with the load balancing mechanics that implies) or the connection never moves at all.

The second consequence is fan-out. The moment two clients on two different servers need the same event (every chat after the first user, every shared document after its first editor) a server-to-server channel must carry events from the machine that received them to every machine holding a listener. That channel is a message queue or an event stream doing pub/sub duty: one event in, every subscribed server out. Skip it and the chat works in single-server testing and fails in production at the exact moment it acquires a second server; build it and the backplane becomes the system’s real backbone, with the delivery-guarantee questions that implies, because websockets themselves guarantee nothing about redelivery.

The third consequence is resource shape. Idle websocket connections are cheap (the memory for a connection is small) but active fleets are not symmetrical with request-response work: a websocket server spends its life holding thousands of long-lived file descriptors, pinging quietly, and writing whenever events arrive. The practical numbers move with runtime, hardware, and workload, so this article will not fabricate benchmarks, but the shape matters: thread-per-connection servers die here, which is the pass-through-or-bypass dilemma from the gateway article wearing a different hat, and event-loop runtimes live here. Plan for descriptors and memory per connection, not requests per second.

The fourth consequence is flow. A slow client on a fast stream backs up: the server either buffers without bound (memory death) or drops and coalesces, and the honest answer is the bounded one: per-connection buffers with limits, messages dropped or merged when the limit is hit, and the reconnecting client resynchronizing from state rather than from a replayed queue. The backpressure article prices the pattern for consumers in general; here it is enough to say that every serious websocket stack rediscovers it, usually as an incident.

The routing consequence deserves the last word, because it changes what the balancer must be able to see. Pinning a connection by source IP works until every user sits behind the same NAT: one office, one IP, one server, one very bad evening. Pinning by a cookie the balancer itself sets works better, but it requires the balancer to speak HTTP, to terminate the connection and read the request headers, which is exactly the l4 vs l7 load balancing distinction the cluster’s comparison article draws: where the balancer looks determines what it can pin. Real-time fleets almost always end up on the L7 side for exactly this reason.

Common mistakes

• No heartbeats. TCP will not tell you a connection is dead until you try to use it, half-open connections accumulate silently until a NAT timeout kills a live one. Ping on an interval, expect pongs, and close on silence: the protocol’s control frames exist precisely for this.
• No reconnect discipline. Clients that reconnect in a tight loop after a server blip turn one outage into a retry storm, thousands of clients hammering the handshake at once. Reconnect with exponential backoff and jitter is not optional; the retry with exponential backoff article is the pattern’s full treatment.
• Assuming delivery guarantees. The protocol delivers frames while the connection lives and says nothing about what crosses a disconnect. Messages sent during a drop are gone; the reconnect must resynchronize from server state: sequence numbers, versioning, or a resync endpoint. Teams that skip this ship chat that loses messages, then rediscover at-least-once delivery and the idempotency machinery it requires.
• Forgetting the infrastructure chain. The handshake must survive every proxy and balancer between client and server: Upgrade headers forwarded, buffering off, timeouts stretched. Websocket bugs that reproduce nowhere in development are usually infrastructure in between: test the chain, not just the endpoints.
• Using websockets for everything. A connection that is full-duplex, stateful, and uncacheable is the wrong tool for request-response work: the server pushing one notification per hour over a websocket fleet has built the most expensive cron in the building. If the data flows one way, the push-strategy comparison in the next article is the honest answer; if it flows rarely, plain HTTP still works.

FAQ

When should I use websockets instead of HTTP?
When the conversation is genuinely two-way and continuous: chat, collaborative editing, multiplayer, live market data. If the server mostly talks and the client mostly listens, the HTTP-native push strategies compared in the next article are simpler; if the client asks and the server answers, HTTP was never broken.

Do websockets work with HTTP/2?
Not by the ordinary Upgrade path; the websocket handshake is defined over HTTP/1.1, and running it inside HTTP/2 requires Extended CONNECT (RFC 8441). Browsers support it; infrastructure support is uneven, and proxies are often happier forwarding a plain upgrade than a CONNECT-shaped one. Treat HTTP/2 websockets as supported-but-verify.

How do I scale websocket connections?
Three pieces: sticky or per-connection routing so a connection stays with its server, a pub/sub backplane (a message queue or stream) so events reach every server holding a listener, and backpressure so slow clients cannot eat memory. Then budget descriptors and memory per connection: the unit of scale is the connection, not the request.

What happens when a websocket connection drops?
The client learns the connection is gone; quickly, if heartbeats are configured. Messages sent after the drop were never delivered; the application protocol must resynchronize: catch up from state, sequence numbers, or a resync endpoint. Then reconnect with backoff, not with a hammer.

Are websockets secure?
As secure as the surrounding decisions. wss runs the connection inside TLS; the browser handshake carries an Origin header the server should check; and authentication still has to happen; the upgrade is an HTTP request and can carry cookies or tokens like any other. The risks are the ordinary ones: authenticate the upgrade, validate the origin, and never trust the client to enforce permissions.

Related articles

• Next read: long polling vs sse; the HTTP-native push strategies and the choosing question this article kept deferring: when the server talks more than it listens, long polling and server-sent events may be the honest answer.
• API gateway; the pass-through-or-bypass dilemma from the other side: what a request-response gateway does when worker threads meet idle connections.
• load balancing in system design, why per-request freedom ends at a persistent connection: algorithms and health checks built for stateless fleets.
• backpressure; the flow-control pattern for the slow-client problem: bounded queues and what to do when the bound is hit.
• message queues in system design; the backplane half of the scaling story: delivery guarantees and the fan-out that carries one event to every server holding a listener.

Last updated on 21 September 2026