IPsec VPN: The Complete Guide

IPsec is the encryption layer most enterprise WANs quietly depend on. Every site-to-site VPN, every DMVPN spoke, every SD-WAN overlay tunnel lands on the same two protocols: IKE to negotiate the keys, and ESP to carry the encrypted payload. Get them subtly wrong and you get the failure that has burned every network engineer at least once, Phase 1 up and Phase 2 dead, with a tunnel that looks healthy in one show command and passes zero packets in another.

This is the hub for the PingLabz IPsec cluster: what IPsec protects, how IKE Phase 1 and Phase 2 fit together, the six ways Cisco lets you build a tunnel, the transform sets to use in 2026, a minimum viable config on IOS XE, and a troubleshooting method built on real error codes. Every line of CLI output here came out of a live Cisco Modeling Labs topology.

What IPsec Solves

The internet is a shared medium you do not control. Anything you send across it is visible to every device on the path: the inner host addresses, the routing protocol you run over the link, the payload. IPsec fixes three things at once.

• Confidentiality. ESP encrypts the payload, so the transit provider sees ciphertext.
• Integrity. An HMAC over each packet means a man in the middle cannot flip a bit unnoticed, or inject packets your peer will accept.
• Peer authentication. Both ends prove who they are (pre-shared key or certificate) before traffic flows. You are not just encrypting to whoever answered on that IP.

In the PingLabz IPsec lab, EDGE1 (203.0.113.1) and EDGE2 (198.51.100.2) sit either side of ISP1. Neither private LAN is routable on the internet, and ISP1 confirms it:

ISP1#show ip route
Gateway of last resort is not set
C        198.51.100.0/30 is directly connected, Ethernet0/1
C        203.0.113.0/30 is directly connected, Ethernet0/0

ISP1#ping 10.30.10.1 repeat 3
...
Success rate is 0 percent (0/3)
ISP1#ping 192.168.99.100 repeat 3
...
Success rate is 0 percent (0/3)

The ISP has no route to 10.30.10.0/24 or to the 192.168.99.0/24 LAN behind bridge1. The only thing making those subnets reach each other is the tunnel. That is the point of a VPN: private reachability across infrastructure that has no idea your networks exist.

How IPsec Works: IKE Phase 1 and Phase 2

IPsec is two conversations stacked on each other, and most confusing IPsec bugs come from mixing them up.

Phase 1 (IKE SA) builds a secure management channel between the two public IPs. It negotiates encryption, hash, authentication method and a Diffie-Hellman group, runs DH, and authenticates the peer. Success gives you an ISAKMP SA in state QM_IDLE. Nothing of yours is encrypted yet. Phase 1 protects the negotiation, not your data.

Phase 2 (IPsec SA) runs inside that channel and negotiates the data SAs: the transform set (what ESP will actually do), the proxy identities (which traffic gets encrypted), and optionally PFS, a fresh DH exchange so a compromised Phase 1 key cannot retroactively decrypt data. It produces two unidirectional SAs, each with its own SPI.

That split explains the most common IPsec symptom in the field. A peer address or PSK problem breaks Phase 1. A crypto ACL or transform-set problem leaves Phase 1 healthy and kills Phase 2 alone. If you take one thing from this page, take that. For the full walk through of both exchanges, read IKE Phase 1, Phase 2, and what actually gets encrypted.

The IPsec Toolbox

Cisco gives you six ways to apply IPsec, all on the same IKE and ESP underneath. What differs is how you select traffic, whether you get a routable tunnel interface, and how far it scales.

For a two-site build: use SVTI unless something forces you not to. The full argument, including MTU, QoS and multicast behaviour, is in crypto maps vs VTI: which site-to-site design to pick.

Transform Sets and Modern Crypto

A transform set answers "what will ESP actually do to this packet". Here is what to use in 2026.

Deprecated, and here is why. If esp-3des, esp-md5-hmac or group 2 appear in your configs, treat them as findings, not options. 3DES has a 64-bit block size, exposing long-lived connections to birthday-bound collision attacks (Sweet32). MD5 is broken for collision resistance. DH group 2 is a 1024-bit MODP group, inside precomputation range for a funded adversary (the Logjam class of attack), and it is the weakest link no matter how strong your ESP cipher is. The only reason to configure any of them is a peer that cannot do better, and even then it is a ticket, not a design.

IKEv2 and FlexVPN

Everything above assumes IKEv1, because that is what most brownfield configs still run. IKEv2 is not IKEv1 with a bigger version number: it is a separate protocol with its own RFC, exchanges, and state machine. Four messages bring up the IKE SA and the first child SA, where IKEv1 needs main mode plus quick mode. Dead peer detection and anti-DoS cookies are built in, NAT traversal is part of the protocol rather than bolted on, and authentication is asymmetric (one side can use a pre-shared key, the other a certificate). The full breakdown of how IKEv2 differs from IKEv1 is worth reading before you convert anything.

The practical win is the smart defaults. IOS XE ships a default IKEv2 proposal, policy, and IPsec profile, so a working tunnel needs a keyring, a profile, and an interface. You can still override every knob, but you no longer type thirty lines to say "AES, SHA, group 14". Two builds matter: the policy-based IKEv2 tunnel using crypto maps when you are grafting IKEv2 onto an existing design, and the route-based IKEv2 tunnel over a static VTI, which is what you should actually deploy on anything new.

FlexVPN goes further and uses IKEv2 itself as the control plane. The IKEv2 authorization policy pushes the tunnel address, the DNS servers and, crucially, the routes to the peer during the exchange, so you run no IGP over the overlay. One config model then covers site-to-site, hub-and-spoke, and remote access, with virtual-access interfaces cloned per peer. Start with what the FlexVPN framework actually is, then the two-router FlexVPN build on IOS XE, then dynamic spoke-to-spoke tunnels without DMVPN.

Minimum Viable Site-to-Site VPN on Cisco IOS XE

The classic IKEv1 crypto-map build, still what you meet on the far side of most third-party VPNs. EDGE1 is a cat8000v on IOS XE 17.18.02 at 203.0.113.1, peering with EDGE2 at 198.51.100.2. We want 10.20.10.0/24 and the 192.168.99.0/24 bridge1 LAN (home of the Debian VM) to reach 10.30.10.0/24.

crypto isakmp policy 10
 encryption aes 256
 hash sha256
 authentication pre-share
 group 14
 lifetime 3600
crypto isakmp key PingLabz-L2L-PSK-01 address 198.51.100.2
crypto ipsec transform-set PLZ-TS-AES256 esp-aes 256 esp-sha256-hmac
 mode tunnel
ip access-list extended PLZ-CRYPTO-ACL
 permit ip 192.168.99.0 0.0.0.255 10.30.10.0 0.0.0.255
 permit ip 10.20.10.0 0.0.0.255 10.30.10.0 0.0.0.255
crypto map PLZ-CMAP 10 ipsec-isakmp
 set peer 198.51.100.2
 set transform-set PLZ-TS-AES256
 set pfs group14
 match address PLZ-CRYPTO-ACL
interface GigabitEthernet2
 crypto map PLZ-CMAP

Now the part that trips people on day one. A crypto map tunnel is built on demand, and the first packets are consumed triggering IKE:

EDGE1#ping 10.30.10.1 source Loopback10 repeat 5
.....
Success rate is 0 percent (0/5)

That is not a failure, that is IKE negotiating. Ping again and the tunnel is there:

EDGE1#ping 10.30.10.1 source Loopback10 repeat 10
Packet sent with a source address of 10.20.10.1
!!!!!!!!!!
Success rate is 100 percent (10/10), round-trip min/avg/max = 10/21/48 ms

EDGE1#show crypto isakmp sa
IPv4 Crypto ISAKMP SA
dst             src             state          conn-id status
198.51.100.2    203.0.113.1     QM_IDLE           1001 ACTIVE

QM_IDLE means Phase 1 is established and idle, waiting to service Quick Mode. It says nothing about whether data is flowing. For that you need Phase 2:

   local  ident (addr/mask/prot/port): (10.20.10.0/255.255.255.0/0/0)
   remote ident (addr/mask/prot/port): (10.30.10.0/255.255.255.0/0/0)
   current_peer 198.51.100.2 port 500
     PERMIT, flags={origin_is_acl,}
    #pkts encaps: 9, #pkts encrypt: 9, #pkts digest: 9
    #pkts decaps: 9, #pkts decrypt: 9, #pkts verify: 9

     local crypto endpt.: 203.0.113.1, remote crypto endpt.: 198.51.100.2
     PFS (Y/N): Y, DH group: group14

     inbound esp sas:
        transform: esp-256-aes esp-sha256-hmac ,
        in use settings ={Tunnel, }
        Status: ACTIVE(ACTIVE)

Read that like a checklist. The proxy IDs are what the two ends had to agree on, and the encap and decap counters are the only honest answer to "is traffic being encrypted". flags={origin_is_acl,} is the crypto map admitting it derived those identities from your ACL, which is exactly what a VTI does not do.

The crypto ACL had two lines, so you get two proxy-ID pairs, each with its own SA pair. The second carries the real Debian VM across the encrypted WAN:

j@llmbits:~$ ping -c 5 10.30.10.1
64 bytes from 10.30.10.1: icmp_seq=1 ttl=254 time=2.82 ms
64 bytes from 10.30.10.1: icmp_seq=2 ttl=254 time=2.41 ms
5 packets transmitted, 5 received, 0% packet loss, time 4007ms

The full command-by-command build, both sides, is in building a site-to-site IPsec VPN with crypto maps on IOS XE.

Why GRE and IPsec Belong Together

This is the section to remember. A crypto map only encrypts what its ACL matches, and an ACL cannot match multicast in a way that lets a routing protocol form an adjacency. So people reach for GRE, which happily carries OSPF, EIGRP and multicast. And it works:

EDGE1#show ip ospf neighbor
Neighbor ID     Pri   State           Dead Time   Address         Interface
10.255.0.2        0   FULL/  -        00:00:35    10.0.0.2        Tunnel0

EDGE1#show interface Tunnel0
Tunnel0 is up, line protocol is up
  Tunnel source 203.0.113.1 (GigabitEthernet2), destination 198.51.100.2
  Tunnel protocol/transport GRE/IP
  Tunnel transport MTU 1476 bytes

EDGE1#show crypto ipsec sa
   (empty - NOTHING is encrypted)

OSPF is FULL, traffic flows, and show crypto ipsec sa is empty, because GRE is encapsulation, not encryption. It does nothing for confidentiality. Here is what ISP1 sees on the link while the VM pings across that plain GRE tunnel:

No.  Time      Source           Destination      Proto  Info
1    0.000000  10.0.0.2         224.0.0.5        OSPF   LS Acknowledge
2    1.461513  192.168.99.100   10.30.10.1       ICMP   Echo (ping) request  id=0x004e, seq=1/256, ttl=63
3    1.462810  10.30.10.1       192.168.99.100   ICMP   Echo (ping) reply    id=0x004e, seq=1/256, ttl=255
4    2.463253  192.168.99.100   10.30.10.1       ICMP   Echo (ping) request  id=0x004e, seq=2/512, ttl=63
5    2.464559  10.30.10.1       192.168.99.100   ICMP   Echo (ping) reply    id=0x004e, seq=2/512, ttl=255
8    4.100454  10.0.0.1         224.0.0.5        OSPF   Hello Packet
15   7.206880  10.0.0.2         224.0.0.5        OSPF   Hello Packet

Look at what the transit provider just learned for free. Your inner host addresses. Your internal routing protocol, hellos included, which leaks your topology. Your traffic patterns. The payload is in the clear too, ICMP simply has nothing interesting in it. A GRE tunnel carrying HTTP or SMB would be a full-content capture.

Now add four lines of tunnel protection, change nothing else, and re-run the same ping at the same point:

crypto ipsec profile PLZ-IPSEC-PROF
 set transform-set PLZ-TS-AES256
 set pfs group14
interface Tunnel0
 tunnel protection ipsec profile PLZ-IPSEC-PROF

No.  Time      Source         Destination    Proto   Info
3    5.153552  203.0.113.1    198.51.100.2   ISAKMP  Identity Protection (Main Mode)
6    5.617148  198.51.100.2   203.0.113.1    ISAKMP  Identity Protection (Main Mode)
9    5.861495  203.0.113.1    198.51.100.2   ISAKMP  Quick Mode
11   6.425161  203.0.113.1    198.51.100.2   ISAKMP  Quick Mode
12   7.026032  203.0.113.1    198.51.100.2   ESP     ESP (SPI=0x6d4428aa)
13   8.939238  198.51.100.2   203.0.113.1    ESP     ESP (SPI=0x94cd6141)
14  12.002692  203.0.113.1    198.51.100.2   ESP     ESP (SPI=0x6d4428aa)
15  12.003690  198.51.100.2   203.0.113.1    ESP     ESP (SPI=0x94cd6141)
16  13.004286  203.0.113.1    198.51.100.2   ESP     ESP (SPI=0x6d4428aa)
17  13.005508  198.51.100.2   203.0.113.1    ESP     ESP (SPI=0x94cd6141)

The ISAKMP exchange at the top is the handshake. After that, every packet is ESP between two public IPs. No inner addresses. No OSPF. No topology. The observer knows two routers are talking and nothing else. That before-and-after justifies the entire protocol suite, and it took four lines. Full walkthrough in GRE over IPsec: why tunnels and encryption belong together.

The next step is to drop GRE entirely. One line, tunnel mode ipsec ipv4, turns Tunnel0 into a native IPsec interface with no GRE header and no crypto ACL:

EDGE1#show interface Tunnel0
  Tunnel protocol/transport IPSEC/IP          <-- was GRE/IP
  Tunnel transport MTU 1438 bytes             <-- was 1476 (no 4-byte GRE header)

EDGE1#show crypto session detail
  IPSEC FLOW: permit ip 0.0.0.0/0.0.0.0 0.0.0.0/0.0.0.0     <-- ANY/ANY. No crypto ACL.
        Inbound:  #pkts dec'ed 18 drop 0
        Outbound: #pkts enc'ed 14 drop 0

OSPF stays FULL, encryption stays on, the proxy IDs go any-to-any, and the crypto ACL disappears from your life. That is Static VTI, a six-line interface config.

Scaling to Many Sites

Everything above is point-to-point. The arithmetic is brutal: a full mesh of n sites needs n(n-1)/2 tunnels, so 20 branches means 190 tunnels to keep in sync. Nobody does that by hand.

DMVPN solves it with mGRE plus NHRP. One tunnel interface on the hub, one on each spoke, and NHRP resolves spoke public addresses on demand so spoke-to-spoke tunnels build themselves. Adding a branch means touching the spoke only. The DMVPN cluster covers phases 1 through 3, NHRP, and per-tunnel QoS. DMVPN is IPsec with a control plane bolted on, which is why this page is a prerequisite for it.

FlexVPN is the IKEv2-native framework: one configuration model for site-to-site, hub-and-spoke and remote access, with virtual-templates cloning an interface per peer. Because IKEv2 pushes routes, spoke-to-spoke tunnels build dynamically without DMVPN or NHRP. Greenfield and large scale, both ends Cisco: FlexVPN is the answer.

GETVPN has no tunnels. A key server distributes a group SA to every member, and members encrypt the payload while preserving the original IP header, so your MPLS provider's routing still works. It only makes sense on a private routed core, never the public internet (a preserved private source IP is not routable there).

NAT, Firewalls, and IPsec

ESP is IP protocol 50 and has no port numbers. PAT devices track flows by port, so a PAT box has nothing to translate on and your ESP packets die. Move EDGE2 behind NAT1's PAT, turn NAT-T off, and watch:

EDGE2(config)#no crypto ipsec nat-transparency udp-encapsulation

EDGE2#ping 192.168.99.100 source Loopback10 repeat 5
.....
Success rate is 0 percent (0/5)

EDGE2#show crypto isakmp sa
dst             src             state          conn-id status
203.0.113.1     10.30.99.2      QM_IDLE           1008 ACTIVE     <-- PHASE 1 IS UP

EDGE2#show crypto ipsec sa | include #pkts
    #pkts encaps: 0, #pkts encrypt: 0, #pkts digest: 0           <-- PHASE 2 IS DEAD
    #pkts decaps: 0, #pkts decrypt: 0, #pkts verify: 0

Phase 1 rides UDP 500, which PAT translates happily. Phase 2 rides raw ESP, which it cannot. Classic "Phase 1 up, Phase 2 down", and this time it has nothing to do with your policy.

NAT Traversal is the fix. The peers detect NAT in the path during IKE and wrap ESP inside UDP 4500, which PAT can track like any other UDP flow. On modern IOS XE it is on by default:

EDGE2#show crypto session detail
Peer: 203.0.113.1 port 4500 fvrf: (none) ivrf: (none)      <-- port 4500, not 500
          Capabilities:N connid:1011 lifetime:00:57:40     <-- N = NAT-traversal

EDGE2#show crypto ipsec sa
     local crypto endpt.: 10.30.99.2, remote crypto endpt.: 203.0.113.1   <-- PRIVATE local endpoint
    #pkts encaps: 15, #pkts encrypt: 15, #pkts digest: 15
        in use settings ={Tunnel UDP-Encaps, }        <-- THE PROOF

NAT1#show ip nat translations
Pro Inside global      Inside local       Outside local      Outside global
udp 198.51.100.6:4790  10.30.99.2:500     203.0.113.1:500    203.0.113.1:500
udp 198.51.100.6:4791  10.30.99.2:4500    203.0.113.1:4500   203.0.113.1:4500

Tunnel UDP-Encaps and Capabilities:N are the strings to grep for, and on the NAT box the ESP is now a translatable UDP flow. The firewall rules follow: permit UDP 500, UDP 4500 and IP protocol 50 between the peers. On a Cisco ASA the same applies, plus NAT exemption so VPN traffic is not translated before it reaches the crypto engine. Deep dive: how NAT-T lets IPsec survive a NAT device.

IPsec on the ASA. The same IKE and ESP mechanics carry over to a Cisco firewall, just with ASA-specific syntax. For a full site-to-site build, follow LAN-to-LAN IPsec VPN on the ASA with IKEv2. And when a tunnel negotiates but passes no traffic through a firewall in the path, work through IPsec through an ASA: NAT-T, ESP pass-through, and why the tunnel won't come up.

Troubleshooting IPsec

The method never changes. Check Phase 1 first with show crypto isakmp sa. If it is not QM_IDLE, the problem is peer-level: peer address, PSK, ISAKMP policy, reachability. If it is QM_IDLE, go straight to the encap and decap counters, because the problem is traffic-level. IOS then names it, as a numeric error code in the ISAKMP-ERROR logs.

Here is error 32 as IOS prints it, worth reading once so you recognise it at 2am:

EDGE2#show logging
Crypto mapdb : proxy_match
map_db_find_best did not find matching map
crypto_mapdb_find_map Fail to find matching policy ivrf 0, fvrf 0, flags 129,
  ike_profile NULL, local_proxy 10.30.10.0, remote_proxy 10.20.10.0,
  km_local 198.51.100.2, km_remote 203.0.113.1
ISAKMP-ERROR: (1018):IPSec policy invalidated proposal with error 32
ISAKMP-ERROR: (1018):phase 2 SA policy not acceptable! (local 198.51.100.2 remote 203.0.113.1)

All five faults, broken deliberately in the lab with full logs, are in troubleshooting IPsec: Phase 1 up, Phase 2 down, and other classics.

The Full IPsec Cluster, in Reading Order

FAQ

Why is my IPsec Phase 1 up but Phase 2 down?

Because they negotiate different things. Phase 1 agrees peer-level parameters and produces the QM_IDLE ISAKMP SA. Phase 2 agrees traffic-level parameters. Non-mirrored crypto ACLs give error 32, mismatched transform sets give error 256, a static entry shadowing a dynamic one gives error 64, and a peer behind PAT with NAT-T off has Phase 1 up while raw ESP is dropped. Check #pkts encaps, not the ISAKMP state.

Do I still need GRE if I use a VTI?

Usually not. An SVTI is already a routable interface carrying IP unicast and multicast, so OSPF and EIGRP form adjacencies over it exactly as over GRE, and you save the 4-byte header (MTU 1438 instead of 1476). You still want GRE for non-IP protocols, for mGRE in DMVPN, or when the far end demands it. Otherwise, SVTI.

Is AES-256 with SHA-256 and DH group 14 still good enough in 2026?

Yes, and it is what the labs on this site use. AES-GCM-256 is better where both peers support it (one pass, no separate HMAC), and ECDH groups 19 or 20 are stronger and cheaper. What is not acceptable is 3DES, MD5 or DH group 2. Turn PFS on regardless of suite.

What ports and protocols do I open on the firewall for a site-to-site VPN?

UDP 500 for IKE, IP protocol 50 for ESP, and UDP 4500 for NAT-T. If either peer sits behind NAT you will only see UDP 500 and 4500 on the wire. Permit all three between the peers and you are covered either way. Do not forget NAT exemption on any device also doing translation, or the proxy IDs stop matching.

Why does the first ping across a new crypto-map VPN always fail?

Because the tunnel is built on demand. The first packets match the crypto ACL, IOS sees no SA, and triggers IKE. Those packets are consumed doing it, so you get a row of dots. In our lab all five packets of the first ping repeat 5 were eaten and the second attempt was 100 percent. A VTI with a routing protocol over it does not do this, the adjacency keeps the SA up.

Key Takeaways

• Phase 1 is peer-level, Phase 2 is traffic-level. Diagnose in that order, and never treat QM_IDLE as evidence data is flowing. The encap and decap counters are.
• GRE encapsulates, IPsec encrypts. Plain GRE hands the ISP your inner host IPs and your OSPF hellos. Four lines of tunnel protection turns it all into ESP between two public IPs.
• Use SVTI by default. Six lines, no crypto ACL, a routable interface. Reach for crypto maps only when a third-party peer forces it.
• Modern crypto only. AES-256 or GCM-256, SHA-256 or better, DH group 14 as the floor, PFS on. 3DES, MD5 and DH group 2 are findings.
• ESP has no ports, so PAT kills it. NAT-T wraps ESP in UDP 4500. Grep for Tunnel UDP-Encaps.
• Learn the error codes. 32 is a crypto ACL mismatch, 64 is a peer not found, 256 is a transform-set mismatch.
• Scale changes the design, not the protocol. DMVPN, FlexVPN and GETVPN all sit on the same IKE and ESP foundation.