Updated Oct 2026: added a packet-capture walkthrough built in my home lab, showing the move from UDP 500 to 4500, ESP inside UDP and NAT keepalives on the wire.
Every IPsec engineer eventually hits the same wall. The tunnel worked perfectly in the lab, you shipped the branch router, and now the site is behind a broadband modem doing PAT. Phase 1 comes up. Phase 2 does not. Nothing crosses. This is one of the most recognisable failures in the whole of IPsec VPN, and the reason for it is embarrassingly simple: ESP has no port numbers, and PAT is a machine that rewrites port numbers. Not every NAT device fails this way, as the packet capture further down shows, but enough do that NAT-T exists.
NAT Traversal (NAT-T) is the fix. This article shows you exactly why raw IPsec can die behind a NAT, what NAT-T actually changes on the wire, and what the failure and the fix look like in real CLI output from a live CML lab, including the capture almost nobody publishes: the NAT device's own translation table.
Why ESP and PAT Often Cannot Coexist
Start with the protocol numbers, because the whole story is in them.
PAT (what Cisco calls NAT overload) multiplexes many inside hosts onto one public address by giving each flow a unique source port. That is its entire mechanism. Hand it an ESP packet and there is no port field to allocate, no port field to record, and no port field to reverse on the way back. Some NAT implementations bolt on an ESP passthrough hack that tracks the SPI instead, and it works for exactly one inside host, until a second one shows up. Many NAT and PAT devices have no such passthrough and simply cannot translate raw ESP, which is why NAT-T exists. When I rebuilt this lab in my home lab in October 2026, the IOS PAT router turned out to have one: it tracked ESP by SPI (the translation entries and hits are in the capture section below), so raw ESP worked for a single peer. NAT-T is still the reliable choice, especially with several peers behind one address or a NAT device that does not handle ESP.
AH is worth a sentence of its own because people ask. AH authenticates the immutable fields of the outer IP header, and the source address is one of them. NAT rewrites that address by definition. The receiver recomputes the ICV, gets a different answer, and drops the packet. There is no NAT-T for AH and there never will be. NAT traversal is an ESP-only story, which is one more reason nobody deploys AH.
The Lab
Two site-to-site IPsec endpoints, EDGE1 at the head end on 203.0.113.1 and EDGE2 at the branch. For this test EDGE2 was moved behind NAT1, an IOS router doing plain PAT on its outside interface:
ip nat inside source list PLZ-PAT interface Ethernet0/0 overloadEDGE2's real address is now a private 10.30.99.2. To the rest of the world, including EDGE1, it appears as 198.51.100.6. EDGE2 itself has no idea any of this is happening, which turns out to matter a great deal.
The Failure: Phase 1 Up, Phase 2 Dead
Disable NAT-T on the branch router (IOS enables it by default, so you have to go out of your way to break it):
EDGE2(config)#no crypto ipsec nat-transparency udp-encapsulationNow ping across the tunnel:
EDGE2#ping 192.168.99.100 source Loopback10 repeat 5
.....
Success rate is 0 percent (0/5)Total failure. Now look at Phase 1:
EDGE2#show crypto isakmp sa
dst src state conn-id status
203.0.113.1 10.30.99.2 QM_IDLE 1008 ACTIVEQM_IDLE. That is the healthy, settled state of an IKEv1 Phase 1 SA. The tunnel's control plane is completely up. Authentication succeeded, DH completed, the ISAKMP SA is ACTIVE. If you stopped troubleshooting here you would swear the VPN was fine.
Now look at Phase 2:
EDGE2#show crypto ipsec sa | include #pkts
#pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0
#pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0Zero in every column. Not a single packet has been encapsulated or decapsulated. The data plane is stone dead. (A fresh rerun in October 2026 on IOL 17.18.2, with one branch behind one IOS PAT router, behaved differently: raw ESP crossed the PAT router and traffic flowed. That capture is in the packet-capture section below.)
Why the split happens exactly here
This is the part worth internalising, because once you see it you will diagnose this failure from across the room.
IKE negotiation rides on UDP port 500. UDP has ports. NAT1's PAT box takes EDGE2's 10.30.99.2:500 packets, allocates a translation, rewrites the source to 198.51.100.6 with some high port, and forwards them. Replies come back, PAT reverses the translation, EDGE2 receives them. Phase 1 completes normally. Phase 2 (Quick Mode) also runs inside IKE on UDP 500, so the IPsec SAs get negotiated too.
Then the router tries to send actual user data, and that data is ESP. IP protocol 50. It arrives at NAT1 with no ports at all. A PAT device that only works with ports has nothing to multiplex on, no way to build a translation entry, and no way to demultiplex the return traffic, so the packets die at the NAT device. That is what the original run of this lab recorded. In the October 2026 rerun, the IOS PAT router built ESP entries keyed by SPI instead, and raw ESP reached the single branch behind it.
So you get a control plane that works and a data plane that does not. Phase 1 up, Phase 2 at zero, with a NAT in the path, is the classic NAT-T signature. Learn those two show commands together and you will never spend an afternoon on this again.
The Fix: Wrap ESP in UDP 4500
Re-enable NAT transparency (or just leave the default alone) and the tunnel comes straight up. Here is what changed.
EDGE2#show crypto session detail
Session status: UP-ACTIVE
Peer: 203.0.113.1 port 4500 fvrf: (none) ivrf: (none)
Capabilities:N connid:1011 lifetime:00:57:40
IPSEC FLOW: permit ip 10.30.10.0/255.255.255.0 192.168.99.0/255.255.255.0
Inbound: #pkts dec'ed 9 drop 0 life (KB/Sec) 4607998/3597
Outbound: #pkts enc'ed 9 drop 0 life (KB/Sec) 4607999/3597Two things to read here. The peer is on port 4500, not 500. And Capabilities:N, where N means NAT-traversal was detected.
Notice what you did not do: you never configured port 4500 anywhere. IOS discovered the NAT by itself. During Main Mode, both peers exchange NAT-D (NAT Discovery) payloads containing hashes of the source and destination IP and port as each side believes them to be. If the hash a peer receives does not match what it computes from the packet it actually got, an address was rewritten in transit, so there is a NAT in the path. Both ends then agree to float the negotiation from UDP 500 to UDP 4500 and to UDP-encapsulate all subsequent ESP. The whole thing is automatic and it happens during Phase 1.
Now the IPsec SA:
EDGE2#show crypto ipsec sa
local crypto endpt.: 10.30.99.2, remote crypto endpt.: 203.0.113.1
#pkts encaps: 15, #pkts encrypt: 15, #pkts digest: 15
transform: esp-256-aes esp-sha256-hmac ,
in use settings ={Tunnel UDP-Encaps, }Two lines carry the whole lesson.
local crypto endpt.: 10.30.99.2 is the private address. EDGE2 has no clue it is being translated. It builds and signs its packets from 10.30.99.2 and hands them to the network. The NAT rewrite happens downstream, outside the router's awareness, which is precisely why the outer header must not be authenticated (and precisely why AH cannot play).
in use settings ={Tunnel UDP-Encaps, } is the clincher. In a normal tunnel you see {Tunnel, }. Here you see Tunnel UDP-Encaps: tunnel mode, with the ESP payload encapsulated in UDP. That string is the definitive proof that NAT-T is engaged. If you only remember one grep from this article, make it that one.
Traffic flows: 15 packets encapsulated, 9 decapsulated, pings at 100 percent.
The NAT Device's View
This is the capture that closes the argument. Log into NAT1, the PAT router in the middle, and ask it what it is actually tracking:
NAT1#show ip nat translations
Pro Inside global Inside local Outside local Outside global
udp 198.51.100.6:4790 10.30.99.2:500 203.0.113.1:500 203.0.113.1:500
udp 198.51.100.6:4791 10.30.99.2:4500 203.0.113.1:4500 203.0.113.1:4500
NAT1#show ip nat statistics
Total active translations: 2 (0 static, 2 dynamic; 2 extended)
Hits: 79 Misses: 0Two entries. Both say udp. One for IKE on 500, one for ESP-in-UDP on 4500. As far as NAT1 is concerned there is no IPsec here at all, just two ordinary UDP conversations between an inside host and an outside host. It allocates ports (4790 and 4791), it builds extended translations, it counts 79 hits and zero misses. This is PAT doing the most boring thing it knows how to do, and that is the entire point.
Compare that with what a port-only PAT device can do with raw ESP: nothing. No port to allocate, no state to keep. (The IOS PAT router in my October 2026 rerun did better and built esp entries keyed by SPI, as the capture section below shows.) The genius of NAT-T is not that it is clever. It is that it makes IPsec look mundane enough for a PAT box to handle.
What NAT-T looks like on the wire (packet capture)
Counters tell you NAT-T happened. A capture shows you how. In October 2026 I rebuilt this scenario in my home lab on Cisco IOL (IOS 17.18.2) with fresh addressing: a head end on 198.18.36.1, a branch router on the private address 10.36.99.2, and an IOS router between them doing PAT to its outside address 198.18.36.6. The tunnel is IKEv1 with AES-256, SHA-256 and DH group 14. I captured on both sides of the PAT router: the inside link, before translation, and the outside link, after it.
The tables below are tshark field output, one packet per line: frame number, relative time, source, destination, IP protocol, UDP source and destination port, ESP SPI, ISAKMP exchange type, the ISAKMP payload types, the next-payload chain, UDP length and the tshark summary.
Main Mode starts on UDP 500, then moves to 4500
Here is the start of the negotiation as the head end sees it, on the outside of the PAT router:
frame.number|frame.time_relative|ip.src|ip.dst|ip.proto|udp.srcport|udp.dstport|esp.spi|isakmp.exchangetype|isakmp.typepayload|isakmp.nextpayload|udp.length|_ws.col.info
1|0.000000000|198.18.36.6|198.18.36.1|17|4790|500||2|1,2,3,13,13,13,13|1,13,0,0,13,13,13,0|176|Identity Protection (Main Mode)
2|0.003338000|198.18.36.1|198.18.36.6|17|500|4790||2|1,2,3,13|1,13,0,0,0|116|Identity Protection (Main Mode)
3|0.005950000|198.18.36.6|198.18.36.1|17|4790|500||2|4,10,13,13,13,20,20|4,10,13,13,13,20,20,0|444|Identity Protection (Main Mode)
4|0.034364000|198.18.36.1|198.18.36.6|17|500|4790||2|4,10,13,13,13,13,20,20|4,10,13,13,13,13,20,20,0|464|Identity Protection (Main Mode)
5|0.061523000|198.18.36.6|198.18.36.1|17|4791|4500||2||5|120|Identity Protection (Main Mode)
6|0.065339000|198.18.36.1|198.18.36.6|17|4500|4791||2||5|104|Identity Protection (Main Mode)Messages 3 and 4 (frames 3 and 4) each carry two payloads of type 20. Those are the NAT-D payloads: each side sends hashes of the addresses and ports it believes the packet uses. The branch sent from 10.36.99.2 port 500, but the head end received it from 198.18.36.6 port 4790, so the hashes cannot match and both ends learn there is a NAT in the path. Frame 5, the very next Main Mode message, already uses destination port 4500.
The same packets on the inside of the PAT router show what the branch actually sent. It used port 500 to port 500, then 4500 to 4500. The PAT router turned those into source ports 4790 and 4791:
frame.number|frame.time_relative|ip.src|ip.dst|ip.proto|udp.srcport|udp.dstport|esp.spi|isakmp.exchangetype|isakmp.typepayload|isakmp.nextpayload|udp.length|_ws.col.info
3|0.007718000|10.36.99.2|198.18.36.1|17|500|500||2|4,10,13,13,13,20,20|4,10,13,13,13,20,20,0|444|Identity Protection (Main Mode)
4|0.037292000|198.18.36.1|10.36.99.2|17|500|500||2|4,10,13,13,13,13,20,20|4,10,13,13,13,13,20,20,0|464|Identity Protection (Main Mode)
5|0.063077000|10.36.99.2|198.18.36.1|17|4500|4500||2||5|120|Identity Protection (Main Mode)
6|0.068174000|198.18.36.1|10.36.99.2|17|4500|4500||2||5|104|Identity Protection (Main Mode)ESP rides inside UDP 4500
After Quick Mode, user traffic shows up as UDP. On the outside link the IP protocol column says 17, not 50, and the ESP SPI sits inside a UDP 4500 datagram:
frame.number|frame.time_relative|ip.src|ip.dst|ip.proto|udp.srcport|udp.dstport|esp.spi|isakmp.exchangetype|isakmp.typepayload|isakmp.nextpayload|udp.length|_ws.col.info
10|0.998480000|198.18.36.6|198.18.36.1|17|4791|4500|0x0e4c994c||||144|ESP (SPI=0x0e4c994c)
11|1.001668000|198.18.36.1|198.18.36.6|17|4500|4791|0xcd2e63b9||||144|ESP (SPI=0xcd2e63b9)On the inside link it is the same SPI pair from the private address, port 4500 to 4500:
frame.number|frame.time_relative|ip.src|ip.dst|ip.proto|udp.srcport|udp.dstport|esp.spi|isakmp.exchangetype|isakmp.typepayload|isakmp.nextpayload|udp.length|_ws.col.info
10|0.999937000|10.36.99.2|198.18.36.1|17|4500|4500|0x0e4c994c||||144|ESP (SPI=0x0e4c994c)
11|1.004378000|198.18.36.1|10.36.99.2|17|4500|4500|0xcd2e63b9||||144|ESP (SPI=0xcd2e63b9)To the PAT router this is just two UDP flows, which is exactly what its translation table showed:
pat-rtr#show ip nat translations
Pro Inside global Inside local Outside local Outside global
udp 198.18.36.6:4790 10.36.99.2:500 198.18.36.1:500 198.18.36.1:500
udp 198.18.36.6:4791 10.36.99.2:4500 198.18.36.1:4500 198.18.36.1:4500NAT keepalives: one byte every 10 seconds
The branch had crypto isakmp nat keepalive 10 configured. With the tunnel idle, this is what crossed the PAT router:
frame.number|frame.time_relative|ip.src|ip.dst|ip.proto|udp.srcport|udp.dstport|esp.spi|isakmp.exchangetype|isakmp.typepayload|isakmp.nextpayload|udp.length|udp.payload|_ws.col.info
1|0.000000000|10.36.99.2|198.18.36.1|17|4500|4500|||||9|ff|NAT-keepalive
2|10.002043000|10.36.99.2|198.18.36.1|17|4500|4500|||||9|ff|NAT-keepalive
3|20.002213000|10.36.99.2|198.18.36.1|17|4500|4500|||||9|ff|NAT-keepaliveframe.number|frame.time_relative|ip.src|ip.dst|ip.proto|udp.srcport|udp.dstport|esp.spi|isakmp.exchangetype|isakmp.typepayload|isakmp.nextpayload|udp.length|udp.payload|_ws.col.info
1|0.000000000|198.18.36.6|198.18.36.1|17|4791|4500|||||9|ff|NAT-keepalive
2|10.002506000|198.18.36.6|198.18.36.1|17|4791|4500|||||9|ff|NAT-keepalive
3|20.002115000|198.18.36.6|198.18.36.1|17|4791|4500|||||9|ff|NAT-keepaliveEach keepalive is a single ff byte (UDP length 9: an 8-byte header plus one byte), sent from the branch every 10 seconds on the existing 4500 flow. On the outside they leave from the same translated port, 4791, which is the point: the traffic keeps that PAT entry alive while no user data is flowing. In this capture only the branch, the side behind the NAT, sent them.
What happened when I turned NAT-T off
I then disabled NAT-T on the branch with the same command used earlier in this article, no crypto ipsec nat-transparency udp-encapsulation, cleared the SAs and the PAT translations, and sent traffic again. I expected the failure described above. On this IOL build it did not happen.
Without NAT-T, Main Mode messages 3 and 4 carry no type 20 payloads, so neither side looks for a NAT and everything stays on UDP 500. The data then goes as raw ESP, IP protocol 50, with no ports at all, and it went in both directions across the PAT router:
frame.number|frame.time_relative|ip.src|ip.dst|ip.proto|udp.srcport|udp.dstport|esp.spi|isakmp.exchangetype|isakmp.typepayload|isakmp.nextpayload|udp.length|_ws.col.info
3|0.005412000|198.18.36.6|198.18.36.1|17|4790|500||2|4,10,13,13,13|4,10,13,13,13,0|372|Identity Protection (Main Mode)
4|0.031651000|198.18.36.1|198.18.36.6|17|500|4790||2|4,10,13,13,13,13|4,10,13,13,13,13,0|392|Identity Protection (Main Mode)
5|0.058788000|198.18.36.6|198.18.36.1|17|4790|500||2||5|116|Identity Protection (Main Mode)
...
10|1.000432000|198.18.36.6|198.18.36.1|50|||0x0e50d30a|||||ESP (SPI=0x0e50d30a)
11|1.004051000|198.18.36.1|198.18.36.6|50|||0x118ef397|||||ESP (SPI=0x118ef397)The inside capture shows the return ESP (SPI 0x118ef397) delivered to the private branch address:
frame.number|frame.time_relative|ip.src|ip.dst|ip.proto|udp.srcport|udp.dstport|esp.spi|isakmp.exchangetype|isakmp.typepayload|isakmp.nextpayload|udp.length|_ws.col.info
10|0.999553000|10.36.99.2|198.18.36.1|50|||0x0e50d30a|||||ESP (SPI=0x0e50d30a)
11|1.999665000|10.36.99.2|198.18.36.1|50|||0x0e50d30a|||||ESP (SPI=0x0e50d30a)
12|2.004551000|198.18.36.1|10.36.99.2|50|||0x118ef397|||||ESP (SPI=0x118ef397)The PAT router built ESP translations of its own, with SPIs where the port columns would normally be, and recorded no misses:
pat-rtr#show ip nat translations
Pro Inside global Inside local Outside local Outside global
esp 198.18.36.6:0 10.36.99.2:0 198.18.36.1:0 198.18.36.1:118EF397
udp 198.18.36.6:4790 10.36.99.2:500 198.18.36.1:500 198.18.36.1:500
esp 198.18.36.6:0 10.36.99.2:E50D30A 198.18.36.1:0 198.18.36.1:0
Hits: 113 Misses: 0On the branch, the IPsec SA counted traffic both ways and showed plain tunnel mode, without UDP encapsulation, and a host ping across the tunnel got all five replies:
br-gw#show crypto ipsec sa
#pkts encaps: 32, #pkts encrypt: 32, #pkts digest: 32
#pkts decaps: 31, #pkts decrypt: 31, #pkts verify: 31
in use settings ={Tunnel, }PING 10.36.10.10 (10.36.10.10): 56 data bytes
5 packets transmitted, 5 packets received, 0% packet lossSo in this lab, with one IPsec peer behind one IOS PAT router, raw ESP worked: the PAT router tracked the ESP flow by SPI and sent the replies back to the only branch behind it. I did not test several IPsec peers behind one public address, other NAT platforms, or a firewall in the path that only permits UDP, so this capture cannot show where raw ESP breaks. What it does show is why NAT-T is the safer default. With it on, the NAT device only has to handle ordinary UDP; without it, the tunnel depends on that device understanding ESP.
The Costs and the Fine Print
8 bytes of overhead, and your MTU
UDP encapsulation adds a UDP header: 8 bytes on every single ESP packet, on top of the ESP overhead you already pay. That is 8 bytes less room for user data before the packet needs fragmenting. If you were already close to the edge (GRE over IPsec over a 1500-byte path is a classic squeeze), NAT-T can be the thing that tips you into fragmentation, and fragmented ESP is a well-known source of intermittent, size-dependent, maddening failures. Fix it at the source: set ip tcp adjust-mss on the LAN-facing interface and lower the tunnel MTU rather than letting the path fragment for you.
NAT keepalives
PAT translations age out. A NAT device holds a UDP entry for a while after the last packet (often 30 to 300 seconds) and then reclaims the port. If your VPN goes quiet for longer than that, the entry vanishes, and the far end's next inbound packet arrives at a NAT device that no longer knows where to send it. The tunnel appears up on both routers and traffic silently dies until the branch happens to send something first. IPsec NAT keepalives exist purely to prevent this: a tiny periodic packet on UDP 4500 whose only job is to keep the translation warm. On IOS that is crypto isakmp nat keepalive 20. Set it. It is cheap, and the failure it prevents is nasty and intermittent.
Firewall rules: 500 AND 4500
The classic operational mistake. Someone opens UDP 500 on the perimeter firewall because "IPsec uses 500", the tunnel negotiates beautifully, and then no data flows because ESP-in-UDP on 4500 is being dropped. It looks exactly like the failure at the top of this article, and the symptom is identical: Phase 1 up, Phase 2 at zero. Permit UDP 500 and UDP 4500 in both directions. If any peer might sit behind a NAT, you need both, always. (And if no NAT is involved, you also need IP protocol 50 permitted for raw ESP.)
Aggressive Mode, IKEv2, and the rest
NAT-D is exchanged in messages 3 and 4 of Main Mode, which is why the float to 4500 happens mid-Phase-1. IKEv2 does the same job with NAT_DETECTION_SOURCE_IP and NAT_DETECTION_DESTINATION_IP notify payloads in the initial exchange, and the resulting behavior is the same: float to 4500, UDP-encapsulate ESP. The mechanism differs, the concept does not.
Diagnosing It in the Field
Means: Phase 1 is fine. IKE crossed the NAT on UDP 500. Do not stop here.
Means: Phase 2 negotiated but nothing crosses. With a NAT in path, suspect NAT-T immediately.
port 4500 and Capabilities:N.Means: NAT detected and NAT-T active. Port 500 plus a NAT in path means it is not.
in use settings ={Tunnel UDP-Encaps, }.Means: ESP is inside UDP. This is the definitive proof.
udp entries, :500 and :4500.Means: PAT is tracking both flows. No 4500 entry means ESP never got wrapped.
Means: the router is behind a NAT and does not know it. Normal, and exactly why NAT-T exists.
One more warning from the same lab. When we moved EDGE2 behind PAT, the head end kept failing Phase 2 with error 64 and peer address 198.51.100.6 not found, because EDGE1 still had a static crypto map entry pointing at EDGE2's old public address. IOS matched the static entry on proxy ID, found the peer did not match, and rejected the proposal instead of falling through to the dynamic map. If you migrate a peer behind a NAT, clean up the stale static entry. There is more on reading those error codes in our guide to troubleshooting IPsec VPN tunnels.
Key Takeaways
- ESP is IP protocol 50 and has no port numbers. PAT works by rewriting ports, so a port-only PAT device has nothing to multiplex on and cannot track an ESP flow. That single fact is the whole problem. Some, like the IOS PAT router in my October 2026 rerun, track ESP by SPI instead, which worked for a single peer.
- The signature is Phase 1 up, Phase 2 at zero.
show crypto isakmp sareads QM_IDLE whileshow crypto ipsec sashows#pkts encaps: 0, #pkts decaps: 0. IKE on UDP 500 survives PAT; raw ESP often does not. - NAT-T wraps ESP in UDP port 4500, which PAT translates like any other UDP flow. On the NAT box you see two ordinary udp translations, :500 and :4500, and that is the entire trick.
- You do not configure the port. IOS auto-detects the NAT during IKE using NAT-D payloads and floats to 4500 on its own. Confirm it with
Peer: ... port 4500andCapabilities:N. - The proof string is
in use settings ={Tunnel UDP-Encaps, }. Also expectlocal crypto endpt.to show the private address, because the router genuinely does not know it is being translated. - Mind the 8 bytes. The extra UDP header eats MTU. Adjust TCP MSS and tunnel MTU rather than letting the path fragment.
- Enable NAT keepalives so idle PAT entries do not age out and blackhole the tunnel.
- Permit UDP 500 AND UDP 4500. Opening only 500 produces the identical Phase-1-up, Phase-2-dead symptom and wastes hours.
- AH is out. It authenticates the outer IP header that NAT rewrites, so it can never traverse a NAT. NAT-T is an ESP-only story.
NAT-T is one of the pieces that makes real-world IPsec deployable at all, alongside crypto maps, tunnel interfaces and the failure modes that come with them. For the full picture, start at the IPsec VPN pillar, build the baseline tunnel with site-to-site IPsec crypto maps, and keep the IPsec troubleshooting guide open next to the console.