TCP three-way handshake

Why the handshake exists

The handshake serves three purposes: agree on starting sequence numbers, exchange options, and confirm bidirectional reachability. You cannot skip any of these and still have a useful reliable transport.

Without an agreed ISN, the receiver cannot tell duplicate packets from new ones. Without options exchange, both sides would have to assume worst-case defaults (small MSS, no window scaling, no SACK), crippling throughput. Without bidirectional confirmation, the server has no proof the client can actually receive data and might allocate state for a spoofed source.

The full state machine

The handshake covers the top half. Closing has its own dance (FIN, FIN-ACK, ACK, ACK or simultaneous close), and TIME_WAIT exists to absorb stray retransmits from the old connection.

The three packets in detail

SYN

The client sends a TCP segment with the SYN flag set, a randomly chosen 32-bit ISN, and a set of TCP options.

• Source port: ephemeral, picked by the OS.
• Dest port: 80 for HTTP, 443 for HTTPS, etc.
• Seq: x, the client's ISN.
• Ack: 0 (unused on SYN).
• Flags: SYN.
• Window: client's initial receive window.
• Options: MSS, window scale, SACK permitted, timestamps.

SYN-ACK

The server allocates connection state (a TCB, transmission control block), picks its own ISN y, and replies.

• Seq: y, the server's ISN.
• Ack: x+1, acknowledging the SYN consumed one sequence number.
• Flags: SYN + ACK.
• Window: server's initial receive window.
• Options: server's MSS, window scale (if supported), SACK permitted, timestamps echo.

ACK

The client confirms with a final ACK. This packet can carry data (the SYN itself does not, in standard TCP, but TFO changes this).

• Seq: x+1.
• Ack: y+1.
• Flags: ACK.

After the server receives this third packet, both sides transition to ESTABLISHED. Note: technically the client transitions to ESTABLISHED after sending the third packet, before the server has received it, so the client may start sending data immediately.

Why ISNs are randomized

Originally TCP used a clock-based ISN that incremented by 1 every 4 microseconds. This was guessable and enabled connection hijacking. RFC 6528 mandates ISNs be derived from a secret keyed hash of the connection tuple plus a slow-moving counter.

If you can guess the ISN, you can inject packets into someone else's TCP connection. This was a real attack vector in the 1990s and is why every modern stack randomizes.

Options exchanged in the handshake

The handshake is the only chance to negotiate most TCP options. After ESTABLISHED, you are stuck with what you agreed to.

• MSS (Maximum Segment Size): largest TCP payload the receiver can accept. Computed from MTU - IP header - TCP header. Typically 1460 on Ethernet.
• Window Scale: shifts the 16-bit window field left by N bits, supporting windows up to 1 GB. Without this, modern bandwidth-delay products cap a flow well below link capacity.
• SACK Permitted: enables selective acknowledgments, so the receiver can tell the sender exactly which segments are missing instead of forcing retransmit of everything after the loss.
• Timestamps: every segment carries a timestamp, enabling accurate RTT measurement and Protection Against Wrapped Sequence numbers (PAWS).

The 1 RTT cost and what it means

The handshake costs exactly 1 RTT. On a 50 ms link, that is 50 ms before your HTTP request can start. With TLS 1.3 on top (1 more RTT), total cold-start before first request byte is 100 ms. On mobile with 200 ms RTT, it is 400 ms.

This is why HTTP/1.1 keep-alive, HTTP/2 connection multiplexing, and HTTP/3 connection coalescing exist. Reusing a warm TCP connection avoids the handshake entirely.

TCP Fast Open

TFO (RFC 7413) lets the client send data in the SYN packet on repeat connections.

First connection: normal handshake. Server issues a cookie in a TCP option.

Subsequent connections: client sends SYN with the cookie and data. Server validates the cookie and accepts the data immediately, replying with SYN-ACK + response data. The client receives response in roughly 1 RTT total, saving the handshake delay.

TFO adoption has been limited because some middleboxes (NAT, firewalls) strip the cookie option or drop packets with data in the SYN. Linux supports it; many networks do not pass it through reliably.

QUIC's 0-RTT achieves the same goal more reliably because QUIC runs over UDP, which middleboxes do not inspect as closely.

SYN flood attacks and SYN cookies

A SYN flood is a DoS attack: the attacker sends SYNs with spoofed source IPs, never completing handshakes. The server allocates a TCB for each, eventually exhausting memory or hitting the listen queue limit.

SYN cookies (RFC 4987) defend against this. Instead of allocating state on SYN receipt, the server encodes the connection details into the ISN it sends in SYN-ACK. The ISN becomes a cookie: a keyed hash of source IP, source port, MSS, and a counter.

When the final ACK arrives, the server validates the cookie by recomputing the hash and checking it matches ack - 1. If it does, the server allocates the TCB at that moment, having stayed stateless during the flood.

The cost: SYN cookies disable some options (window scaling, SACK) because there is no state to remember them in. Used as a fallback under load, not as default.

Simultaneous open

If both peers send SYN to each other at the same time, both transition SYN_SENT, then SYN_RECEIVED, then ESTABLISHED, using a 4-packet exchange instead of 3. Rare but possible in peer-to-peer applications. The state machine handles it.

What can go wrong

Retransmitted SYN

If the SYN is lost, the client retransmits after the SYN-RTO (usually 1 second initially, doubling). On modern Linux, default retries is 6, so a totally unreachable server takes 1+2+4+8+16+32 = 63 seconds to give up. Configurable via tcp_syn_retries.

Dropped SYN-ACK

If the SYN-ACK is lost, the server retransmits. Client just waits for its SYN reply.

SYN-ACK arrives but ACK is dropped

The server stays in SYN_RECEIVED until it retransmits SYN-ACK, hoping for the ACK. The client thinks it is ESTABLISHED and may send data. The data segment carries an ACK that completes the handshake from the server's perspective, so this usually self-heals.

Half-open connections

If the client dies after sending the final ACK without sending data, the server stays in ESTABLISHED forever unless TCP keepalive is enabled (default 2 hours on Linux, way too long). Application-level heartbeats are usually better.

NAT and the handshake

NAT devices track the handshake to set up port mappings. A SYN going out creates a mapping; the SYN-ACK coming back uses it. Most NATs time out an unestablished mapping in 30-60 seconds, so a half-open connection cannot survive NAT for long.

This is why long-running TCP connections through NAT need application-level keepalive every 30-60 seconds.

Performance tips

• Increase initial cwnd (Linux default is 10 segments since kernel 3.0). Bigger cwnd means more data in the first round trip.
• Enable TCP Fast Open if your network supports it.
• Reuse connections: HTTP/1.1 keep-alive, HTTP/2 multiplexing, connection pools in your HTTP client.
• Consider HTTP/3 (QUIC) for 0-RTT on resume.
• Tune tcp_syn_retries and tcp_synack_retries if you want faster failover.

Numbers to memorize

• Handshake cost: 1 RTT, 3 packets.
• TCP Fast Open: saves 1 RTT on repeat.
• Default Linux SYN retries: 6.
• Default Linux SYN-ACK retries: 5.
• TIME_WAIT duration: 2 MSL, typically 60 seconds on Linux.
• Initial cwnd: 10 segments (RFC 6928).
• Initial RTO: 1 second (RFC 6298).