How to Use ike-scan: Test an IPsec VPN Peer and Read NO-PROPOSAL-CHOSEN

A matching AES-256/SHA-256/DH14 proposal got a Main Mode handshake. A wrong DH group, a wrong hash and the right suite from an address with no PSK all got NO-PROPOSAL-CHOSEN, and only debug crypto isakmp told them apart.

ike-scan lab topology: Kali probing an IOS XE IPsec peer on UDP 500 with IKEv1 and IKEv2

ike-scan sends the first IKE message to a VPN peer and prints what comes back: a handshake (your proposal matches), a NO-PROPOSAL-CHOSEN notify (the peer heard you and refused), or nothing. Against a Cisco IOS XE 17.18.2 peer in my home lab, the matching AES-256/SHA-256/DH14 proposal returned Main Mode Handshake returned in 0.088 seconds. A wrong DH group, a wrong hash and the right proposal from an address with no pre-shared key all returned the same NO-PROPOSAL-CHOSEN. Only the router debug told them apart.

sudo ike-scan -M --trans=7/256,4,1,14 192.0.2.1

This guide covers install, the options checked against ike-scan --help, the matching case, NO-PROPOSAL-CHOSEN field by field with the IOS XE debug line for each, Aggressive Mode, an IKEv2 check, and three kinds of silence. Use it on peers you run or are authorized to test. ike-scan stops at the first exchange: it never sends a key or builds a tunnel.

ike-scan lab topology: Kali probing an IOS XE IPsec peer on UDP 500 with IKEv1 and IKEv2
Kali at the configured peer address probes EDGE on UDP/500: matching and mismatched proposals, a source with no key, Aggressive Mode, IKEv2, and three kinds of silence.

Lab setup

Kali sits at 192.0.2.10, the address the head end expects for the remote site, so probes arrive from the configured peer. A second address, 192.0.2.11, shows what a wrong source looks like.

  • Kali Rolling: ike-scan 1.9.6 (package 1.9.5-2), nmap 7.991, tcpdump 4.99.6
  • EDGE: Cisco Catalyst 8000V, IOS XE 17.18.2, 192.0.2.1 on GigabitEthernet2 (WAN), 198.51.100.1 on GigabitEthernet3 (LAN)
  • Remote LAN in the crypto ACL: 203.0.113.0/24. The pre-shared key appears as <PSK>.

Step 1: Install ike-scan

sudo apt install ike-scan
ike-scan --version
ike-scan 1.9.6

Run probes as root or with sudo. ike-scan binds UDP source port 500 by default, because some peers only answer IKE from port 500.

Step 2: Configure the IOS XE peer

A standard IKEv1 crypto map peer. This is the finished config. Aggressive Mode stayed enabled until Step 5, and WAN-IN was added in Step 7.

crypto isakmp policy 10
 encryption aes 256
 hash sha256
 authentication pre-share
 group 14
 lifetime 86400
crypto isakmp key <PSK> address 192.0.2.10
crypto isakmp aggressive-mode disable
!
crypto ipsec transform-set TS-AES256 esp-aes 256 esp-sha256-hmac
 mode tunnel
!
ip access-list extended VPN-TRAFFIC
 10 permit ip 198.51.100.0 0.0.0.255 203.0.113.0 0.0.0.255
!
crypto map CMAP 10 ipsec-isakmp
 set peer 192.0.2.10
 set transform-set TS-AES256
 match address VPN-TRAFFIC
!
ip access-list extended WAN-IN
 5 permit udp host 192.0.2.10 host 192.0.2.1 eq isakmp
 6 permit udp host 192.0.2.10 host 192.0.2.1 eq non500-isakmp
 7 permit esp host 192.0.2.10 host 192.0.2.1
 10 deny udp any any eq isakmp
 20 permit ip any any
!
interface GigabitEthernet2
 ip address 192.0.2.1 255.255.255.0
 ip access-group WAN-IN in
 crypto map CMAP

ISAKMP does not listen until a crypto map is on an interface. With the policy, key and map defined but not applied, ike-scan got 0 returned handshake; 0 returned notify. Applying the map to GigabitEthernet2 logged %CRYPTO-6-ISAKMP_ON_OFF: ISAKMP is ON.

Before policy 10 existed, show crypto isakmp policy listed two built-in defaults, 65513 and 65514 (AES-128, SHA-1, DH14, with RSA signature and pre-shared key). Once policy 10 existed, the router checked offers only against it. An AES-128/SHA-1/DH14 probe got NO-PROPOSAL-CHOSEN.

Step 3: Send the matching proposal

--trans uses the format enc[/keylen],hash,auth,group, so AES-256, SHA2-256, pre-shared key and DH group 14 is 7/256,4,1,14. Turn on debug crypto isakmp on the router to see its side.

ike-scan Main Mode handshake with the matching IOS XE debug and show crypto isakmp sa
ike-scan Main Mode handshake with the matching IOS XE debug and show crypto isakmp sa

How to read the ike-scan side:

  • Main Mode Handshake returned: the peer accepted a transform and returned its SA payload. It is alive on UDP/500, and your proposal overlaps its policy.
  • HDR=(CKY-R=...): the responder cookie. A non-zero value means the peer created state for you.
  • SA=(...): the transform the peer chose. Offer several and this line shows which one won. With --trans=5,2,1,2 --trans=7/256,4,1,14, the router rejected the 3DES transform and accepted the AES-256 one.
  • LifeDuration=28800: ike-scan's default offer. The router's own actual life: 86400 did not cause a rejection, and neither did a --lifetime=172800 probe.

And the router side:

  • found peer pre-shared key matching 192.0.2.10 comes first. IOS XE looks up the key by source address before it checks any transform.
  • atts are acceptable is the router's version of "Handshake returned".
  • MM_SA_SETUP in show crypto isakmp sa is a half-open SA. ike-scan never sends message 3, so the router retransmits and deletes it with Death by retransmission P1. Expect one after every probe. It is not a tunnel.

Step 4: Read NO-PROPOSAL-CHOSEN field by field

Notify message 14 (NO-PROPOSAL-CHOSEN) proves the peer is reachable on UDP/500 and speaks IKEv1. It does not say which field was wrong. Each probe below changes one thing from the working proposal:

Three single-field mismatches and the ike-scan defaults, each with the IOS XE debug error
Three single-field mismatches and the ike-scan defaults, each with the IOS XE debug error
ProbeWhat changedIOS XE debug line
--trans=7/256,4,1,2DH group 2Diffie-Hellman group offered does not match policy!
--trans=7/256,2,1,14SHA-1Hash algorithm offered does not match policy!
--trans=7/128,4,1,14AES-128Proposed key length does not match policy
no --trans8 default transformsEncryption algorithm offered does not match policy! (all 8)
right suite from 192.0.2.11source has no keyNo pre-shared key with 192.0.2.11!

Plain ike-scan 192.0.2.1 offers 3DES or DES, SHA-1 or MD5, and DH group 2 or 1. A hardened peer rejects all eight, so NO-PROPOSAL-CHOSEN from a plain run usually means a healthy peer. Always pass your real policy.

The last row is the one that wastes hours. The right proposal from the wrong address looks exactly like a cipher mismatch:

Matching proposal from an address with no PSK returns NO-PROPOSAL-CHOSEN
Matching proposal from an address with no PSK returns NO-PROPOSAL-CHOSEN

After the failed key lookup, the router logged Preshared authentication offered but does not match policy!. In production this is a remote site behind NAT, or a new public address with no crypto isakmp key. The fix is a key for the address the packets really come from. Test from that address, and read the key lookup line before you touch a policy.

Step 5: Main Mode vs Aggressive Mode on your own peer

Main Mode takes six messages and protects the identities. Aggressive Mode takes three and sends identities and authentication data before an encrypted channel exists. Most site-to-site designs don't need it. Check whether your peer still answers:

sudo ike-scan -M -A --id=branch1.example.net --dhgroup=14 --trans=7/256,4,1,14 192.0.2.1

In Aggressive Mode the DH group must match --dhgroup, which defaults to 2. With Aggressive Mode at its default, the router answered:

Aggressive Mode handshake from IOS XE, then silence after aggressive-mode disable
Aggressive Mode handshake from IOS XE, then silence after aggressive-mode disable
  • Aggressive Mode Handshake returned with the same AES-256 SA.
  • VID lines for Cisco Unity, Dead Peer Detection v1.0 and XAUTH identify the implementation. The Main Mode reply had none.
  • ID(Type=ID_IPV4_ADDR, Value=192.0.2.1) is the router's identity, sent in the clear.
  • The debug showed no pre-shared key based on hostname branch1.example.net!, then local preshared key found. It fell back to the address key.

From 192.0.2.11, which has no key, Main Mode refused, but Aggressive Mode still returned a full handshake. One line fixed it:

crypto isakmp aggressive-mode disable

The same probe then got 0 handshakes and 0 notifies. The router logged %CRYPTO-5-IKMP_AG_MODE_DISABLED: Unable to initiate or respond to Aggressive Mode while disabled. Main Mode still returned its handshake.

Step 6: Check IKEv2 with --ikev2

--ikev2 (-2) sends an IKE_SA_INIT. Per --help it is experimental and "only supports sending the default proposal". --trans does not apply.

ike-scan --ikev2 against IOS XE smart defaults, then against a lab-only SHA-1 proposal
ike-scan --ikev2 against IOS XE smart defaults, then against a lab-only SHA-1 proposal

The router had no IKEv2 profile, only smart defaults, and still answered with NO_PROPOSAL_CHOSEN in an IKEv2 header. So IKEv2 was listening. debug crypto ikev2 printed both sides:

  • Received Policies: ike-scan's fixed proposal. That means AES-CBC-256/128, 3DES or DES, with PRF SHA1 or MD5, integrity SHA96 or MD596, and DH 2, 5 or 14.
  • Expected Policies: the 17.18.2 default, AES-CBC-256 with SHA512 or SHA384 and DH 19, 14 or 21.

The integrity lists share nothing, so a modern IKEv2 peer always refuses ike-scan. To see a match, the lab added a throwaway SHA-1 proposal, then removed it. Default --ikev2 got Notify message 17 (INVALID_KE_PAYLOAD) (The peer's KE payload contained the wrong DH group). Adding --dhgroup=14 got IKEv2 SA_INIT Handshake returned with Prf=HMAC_SHA1 Integ=HMAC_SHA1_96 DH_Group=14. Its vendor IDs decode to CISCO-DELETE-REASON, CISCOVPN-REV-02, CISCO-DYNAMIC-ROUTE and FLEXVPN-SUPPORTED.

So an IKEv2 handshake from ike-scan is a finding: the peer accepts SHA-1. A refusal only proves IKEv2 is up. Check real IKEv2 proposals with the router debug.

Step 7: Separate a filter from a dead peer

ike-scan ignores ICMP. A closed port, a rejecting ACL and a silent drop all print 0 returned handshake; 0 returned notify. Run tcpdump next to it, or nmap, and they separate:

Three kinds of ike-scan silence separated by tcpdump and nmap
Three kinds of ike-scan silence separated by tcpdump and nmap
Peer statetcpdump on Kalinmap -sU -p 500
No crypto map appliedICMP 192.0.2.1 udp port 500 unreachableclosed
deny udp any any eq isakmp inboundICMP host 192.0.2.1 unreachable - admin prohibited filterfiltered
Same ACL plus no ip unreachablesThree probes, no replyopen|filtered

Only two of three probes drew ICMP in the first two cases. IOS XE rate-limits unreachables to one per 500 ms, and ike-scan's retries come 500 ms and 750 ms apart.

The ACL counters confirm it. One ike-scan run is three tries, and the deny entry showed (3 matches). A permit for the peer above it restored the handshake with (1 match):

WAN-IN ACL counters before and after permitting UDP 500 from the peer
WAN-IN ACL counters before and after permitting UDP 500 from the peer

Permit UDP/4500 and ESP from the peer as well, for NAT-T and data. Before the ACL, ike-scan --nat-t got the same handshake on UDP/4500.

ike-scan options reference

Checked against ike-scan --help on 1.9.6.

OptionDefaultWhat it does
-MoffOne payload per line
--trans=enc[/len],hash,auth,group (-a)8 legacy transformsCustom IKEv1 transform. Repeat to offer several. Not for IKEv2.
--auth (-m)1 (PSK)Authentication method. 3 is RSA signature.
-AMain ModeAggressive Mode. Pair with --dhgroup and --id.
--id (-n), --idtype (-y)none, 3 (USER_FQDN)Aggressive Mode identity. Type 1 is an IPv4 address.
--dhgroup (-g)2Key exchange group for Aggressive Mode and IKEv2
--ikev2 (-2)IKEv1IKE_SA_INIT with ike-scan's fixed proposal
--nat-toffRFC 3947 encapsulation on UDP/4500
--sport, --dport500, 500UDP ports. --sport=0 is random.
--bindipsystem choiceSource address. Used here for 192.0.2.11.
--retry, --timeout, --backoff3, 500 ms, 1.5Tries, first wait, wait multiplier
--lifetime (-l)28800Phase 1 lifetime offered
--file (-f)nonePeer list, one per line

--trans values (RFC 2409): encryption 1 DES, 5 3DES, 7 AES with /128 or /256. Hash 1 MD5, 2 SHA-1, 4 SHA2-256. Auth 1 PSK. Group 2 is 1024-bit and 14 is 2048-bit MODP.

ike-scan vs router commands vs nmap

ToolProvesMisses
ike-scanThe peer answers on UDP/500 or 4500 and which IKEv1 proposal it accepts from your sourceThe key, Phase 2, routing, real IKEv2 proposals
show crypto isakmp policyWhat the router is configured to acceptWhether packets arrive, which key a source maps to
debug crypto isakmp / ikev2The failing field, the key lookup, Received vs ExpectedNothing you can't reproduce. Scope it with debug crypto condition peer ipv4 <ip>.
show crypto isakmp saPer-peer state, including probe leftoversWhy a negotiation failed
nmap -sU -p 500 --script ike-versionPort state: open, closed, filteredProposals. Here the script printed only the port state.

Run ike-scan from outside and the debug from inside at the same time.

Gotchas

`NO-PROPOSAL-CHOSEN` with the right suite. Look for No pre-shared key with <ip>!. IOS XE reports a missing key for your source as a proposal failure.

Silence right after a config change. Confirm the crypto map is on the WAN interface and ISAKMP is ON was logged.

The first probe from a new source got no reply. The router logged No pre-shared key with 192.0.2.11! and sent its notify, but ike-scan printed 0/0, and the router ignored the retries for that cookie. A second run 38 seconds later got the notify. Run twice before you trust silence.

`Notify message 17 (INVALID_KE_PAYLOAD)`. Your key exchange group is not in the peer's proposal. Set --dhgroup.

AES without a key length. --trans=7,... matches no AES-256 policy. Write 7/256.

FAQ

Does "Main Mode Handshake returned" mean the VPN is up?

No. The peer accepted your Phase 1 proposal from that address, and nothing more. The key, transform set, crypto ACL and routing are untested. For the next layer, see Cisco IPsec Phase 1 Up, Phase 2 Down.

Why does plain ike-scan always get NO-PROPOSAL-CHOSEN?

Its eight defaults use 3DES or DES, SHA-1 or MD5, and DH group 2 or 1. A hardened peer rejects them all. Pass --trans.

Can ike-scan test an IKEv2 peer?

Only partly. Any IKEv2 notify proves IKEv2 is listening. ike-scan 1.9.6 sends one fixed SHA-1/MD5 proposal, so it cannot confirm a SHA-384 or SHA-512 policy.

How do I tell a firewall drop from a peer that is down?

Capture while you probe. Port unreachable means nothing is listening. Admin prohibited means an ACL rejected it. No reply means a silent drop or a dead path. Then check the ACL counters on the peer.

Is ike-scan safe against a production head end?

A few probes from an authorized host are light: one packet per try, three tries. Each leaves a half-open SA until the router gives up, so don't loop it against a busy box. Scope debugs to one peer.

Not covered: certificate authentication, IKEv2 profiles, and Phase 2 checks.

Key takeaways

  • ike-scan gives three answers: a handshake (accepted from this source), NO-PROPOSAL-CHOSEN (reachable, refused), or 0/0. Always pass --trans with your real policy.
  • NO-PROPOSAL-CHOSEN does not name the field. debug crypto isakmp does, including No pre-shared key with <ip>.
  • ike-scan reports every silence the same way. tcpdump, nmap and the ACL counters split it into closed, filtered and dropped.
  • Use -A to prove Aggressive Mode is off. Use --ikev2 only to see that IKEv2 is listening.

More in this series: sslscan to check TLS versions and ciphers on a management or VPN portal, snmp-check to find leftover SNMPv2c after a v3 migration, and How to Use Wireshark to capture the IKE exchange. Series hub: Linux Networking Commands: The Complete Guide

Read next