Skip to content
back to writing
8 min readpost-quantum-cryptography · tls · networking

The PQC Migration: Why 3KB Signatures Will Break Your Networking Stack

NIST finalized FIPS 204, but swapping out ECDSA isn't just a library update. ML-DSA signatures will fragment your TLS handshakes and drop your packets.

RG
Rahul Gupta
Senior Software Engineer
share

Most teams that adopt post-quantum cryptography discover the real problem the moment they watch their TLS handshakes silently time out. You swap your cryptography libraries for a PQC-compliant fork, generate your new NIST FIPS 204 keys, and assume the math will just work. Then you deploy it behind a standard AWS Application Load Balancer and watch your p99 latency spike to 400ms while packet fragmentation eats your network throughput alive.

In August 2024, NIST finalised FIPS 203, 204, and 205. For regulated sectors like BFSI and fintech, this shifted post-quantum cryptography from a theoretical research paper to a mandatory engineering migration. The threat model driving this is "Harvest Now, Decrypt Later" (HNDL)—nation-states and adversaries recording encrypted traffic today so they can break it when cryptographically relevant quantum computers (CRQCs) arrive.

The mandate is clear: migrate. But the implementation is a trap. The industry has spent two decades optimising networks for RSA and Elliptic Curve cryptography. We built our MTU limits, TCP window sizes, and middlebox buffers around the assumption that cryptographic keys and signatures are small.

Post-quantum signatures are not small. They are massive. And they are going to break your infrastructure.

The 1500-Byte Reality Check

To understand why your network is going to choke, you have to look at the byte counts.

For the last decade, the industry standard for TLS authentication has been ECDSA (specifically the P-256 curve) or Ed25519. An ECDSA P-256 public key is 32 bytes. The resulting signature is 64 bytes.

The new NIST standard for primary digital signatures is FIPS 204, also known as ML-DSA (formerly Dilithium). The lightest variant, ML-DSA-44, has a public key size of 1,312 bytes. The signature it generates is 2,420 bytes.

Standard Ethernet Maximum Transmission Unit (MTU) is 1500 bytes. Once you subtract the IPv4 header (20 bytes) and the TCP header (20 bytes), your Maximum Segment Size (MSS) is 1460 bytes.

With ECDSA, your entire Certificate and CertificateVerify TLS 1.3 messages fit comfortably inside a single TCP segment. With ML-DSA-44, a single signature exceeds the MTU. When you add the server certificate, the intermediate CA certificates (which will also be using ML-DSA), and the handshake overhead, your server is suddenly trying to push 8KB to 10KB of data just to establish a connection.

This guarantees TCP fragmentation.

Where Fragmentation Kills You

In a perfect world, IP fragmentation is handled transparently by the network stack. In the real world of enterprise fintech, fragmentation is a death sentence for latency and reliability.

The Middlebox Massacre Enterprise networks are littered with Deep Packet Inspection (DPI) appliances, Intrusion Detection Systems (IDS), and next-generation firewalls. Many of these middleboxes are configured to aggressively drop fragmented packets to prevent evasion attacks. Even when they do allow fragments, they have to buffer them in memory to reassemble the payload before inspecting it. Pushing 10KB TLS handshakes through an older Palo Alto or Fortinet appliance will spike its CPU and exhaust its state tables, leading to dropped connections and silent timeouts.

The VPN and IPsec Penalty If you are routing traffic between an on-premise data centre and an AWS VPC over a Site-to-Site VPN, your MTU is not 1500. IPsec encapsulation adds overhead. Your effective path MTU is likely closer to 1390 bytes. When a fragmented ML-DSA handshake hits that tunnel, the router has to fragment the packets again, or drop them and send an ICMP "Fragmentation Needed" message. If ICMP is blocked—which it almost always is in strict BFSI environments—you get a black hole.

The QUIC Amplification Trap If you are running modern web infrastructure using HTTP/3 and QUIC, the situation is even worse. QUIC runs on UDP. To prevent attackers from using your server for DDoS reflection attacks, RFC 9000 enforces a strict anti-amplification limit. A server is forbidden from sending more than three times the number of bytes it has received from an unvalidated client.

A typical QUIC ClientHello is around 1,200 bytes. The 3x amplification limit means the server can send a maximum of 3,600 bytes before it must stop and wait for the client to acknowledge receipt.

An ML-DSA certificate chain and signature easily exceeds 5,000 bytes. The server sends the first 3,600 bytes, hits the protocol limit, and halts. It waits for a client ACK, adding a full network round-trip time (RTT) to the handshake. For low-latency ad-tech or high-frequency trading APIs, adding 50ms to every connection establishment is a P1 incident.

The NIST Roster: Pick Your Poison

You might wonder if we can just pick a smaller algorithm. We cannot. The physics of post-quantum mathematics force a trade-off: you can have small keys, small signatures, or fast compute. You cannot have all three.

AlgorithmStandardTypePublic Key SizeSignature / CiphertextCompute Speed
ECDSA (P-256)FIPS 186-5Classical32 B64 BFast
ML-KEM-768FIPS 203PQC Key Exchange1,184 B1,088 BVery Fast
ML-DSA-44FIPS 204PQC Auth1,312 B2,420 BFast
SLH-DSA-128sFIPS 205PQC Auth32 B7,856 BVery Slow
FN-DSA (Falcon)FIPS 206 (Draft)PQC Auth897 B666 BFast (Requires FPU)

Look at the table. FIPS 205 (SLH-DSA, based on SPHINCS+) gives you tiny public keys but hits you with an 8KB signature. It is also incredibly slow, making it useless for high-volume load balancers.

FN-DSA (Falcon) looks like the winner. It has the smallest combined size. But Falcon requires constant-time 64-bit floating-point arithmetic. Hardware Security Modules (HSMs), smart cards, and embedded devices typically lack hardware FPUs. Writing constant-time software floating-point code that doesn't leak timing side-channels is notoriously difficult.

This leaves ML-DSA (FIPS 204) as the default general-purpose signature scheme. We are stuck with 3KB+ signatures.

What You Can Actually Do

You cannot rewrite the laws of physics, but you can engineer around the network constraints. If you are mandated to begin the PQC migration this year, here is the playbook.

1. Decouple Key Exchange from Authentication

This is the most critical architectural distinction you must make.

The "Harvest Now, Decrypt Later" threat model only applies to Key Exchange. If an adversary records your TLS traffic today, they want to use a future quantum computer to extract the symmetric session key and decrypt the payload.

They cannot retroactively forge an authentication handshake. A digital signature only matters at the exact moment the connection is established. By the time a quantum computer exists to break an ECDSA signature, the TLS session will have been closed for a decade.

Therefore, you do not need ML-DSA (FIPS 204) today. You need ML-KEM (FIPS 203) today.

ML-KEM is a Key Encapsulation Mechanism. Its public keys and ciphertexts are around 1.1KB each. While larger than classical equivalents, they fit cleanly into a standard 1500-byte MTU without catastrophic fragmentation.

You should deploy a Hybrid TLS configuration. Use ML-KEM combined with classical X25519 for the key exchange, but retain ECDSA or RSA for the certificate signatures.

If you are running Go 1.23 or later, hybrid key exchange is already built into the standard library. You enable it via the CurvePreferences in your TLS config:

Go
package main

import (
	"crypto/tls"
	"net/http"
)

func main() {
	config := &tls.Config{
		// X25519Kyber768Draft00 implements the hybrid X25519 + ML-KEM key exchange.
		// This protects against Harvest Now, Decrypt Later without bloating the auth chain.
		CurvePreferences: []tls.CurveID{
			tls.X25519Kyber768Draft00,
			tls.X25519,
			tls.CurveP256,
		},
		MinVersion: tls.VersionTLS13,
	}

	server := &http.Server{
		Addr:      ":443",
		TLSConfig: config,
	}
	
	// Start server...
}

2. Enable Jumbo Frames Inside the VPC

For internal service-to-service communication (east-west traffic), you control the network. If you are enforcing mTLS between microservices using a service mesh like Istio or Linkerd, both sides will eventually need to exchange ML-DSA certificates. That means 8KB+ handshakes.

You must enable Jumbo Frames (MTU 9000).

In AWS, EC2 instances and VPC peering connections support 9001 MTU by default, but your operating system interfaces often default to 1500. You need to explicitly configure your host networking to use the larger MTU.

Shell
# Check current MTU
ip link show dev eth0

# Set to Jumbo Frames temporarily (use netplan/NetworkManager for persistence)
sudo ip link set dev eth0 mtu 9000

This ensures that even a 10KB mTLS handshake only requires two TCP segments, eliminating the middlebox fragmentation drops entirely within your private network.

3. Implement Certificate Compression

TLS 1.3 introduced RFC 8879, which allows for certificate compression using algorithms like Brotli or Zstandard.

Because ML-DSA public keys and signatures are essentially random mathematical noise, they do not compress well. However, the rest of the X.509 certificate—the issuer strings, subject alternate names, and extensions—compresses beautifully.

If you must serve ML-DSA certificates to the public internet where you are bound by 1500-byte MTUs, enabling TLS certificate compression at your ingress controllers (like HAProxy or Nginx) will strip a few hundred bytes off the payload. It won't solve the fragmentation entirely, but it might be the difference between spanning three TCP segments instead of four.

The Verdict

Do not swap your entire PKI infrastructure to FIPS 204 (ML-DSA) this year. The internet is fundamentally unprepared to route 3KB signatures efficiently, and you will spend the next six months debugging silent TCP drops and QUIC timeouts.

What I actually do for BFSI platforms today:

  1. Rotate the Key Exchange: Enable hybrid X25519 + ML-KEM on all load balancers, ingress controllers, and internal TLS terminations. This immediately neutralises the "Harvest Now, Decrypt Later" threat.
  2. Keep Classical Auth: Leave the X.509 certificate chain on ECDSA P-256.
  3. Prep the Network: Start an audit of every middlebox, VPN tunnel, and transit gateway in the architecture. Identify hardcoded 1500 MTU limits and begin migrating internal VPC traffic to Jumbo frames.

Wait for the network vendors to update their DPI buffers and QUIC implementations before you touch the authentication layer. Fix the threat today, but don't break your MTU to do it.

Rahul Gupta
share