OSPF Authentication Mismatch Troubleshooting

OSPF authentication keeps rogue routers out of your routing domain, and it is also one of the most common ways to break a perfectly healthy adjacency. The moment the two ends of a link disagree about authentication, incoming hellos are silently discarded, the neighbor relationship drops, and every route learned through that neighbor disappears with it. If you want the bigger picture of how OSPF builds and maintains neighbors before diving into this specific failure, the full OSPF guide maps the whole process from the first hello to a synchronized database.

The good news is that authentication mismatches are among the friendliest OSPF problems to troubleshoot, because IOS tells you exactly what is wrong, provided you run the right debug. In this article you will see both failure modes (wrong authentication type and wrong key) reproduced on real gear, along with the exact log lines that identify each one. All device output comes from a Cisco Modeling Labs topology running IOS XE 17.18: R1 and R2 share a LAN in area 0, and R2-R3 is a /30 link in area 1 running the point-to-point network type.

The two failure modes: type mismatch vs key mismatch

Every OSPF packet carries an authentication type field in its header. Type 0 means null (no authentication at all), type 1 means simple plaintext password, and type 2 means cryptographic authentication (classic MD5 with message-digest keys, or SHA variants if you use key chains). For an adjacency to form, both routers must first agree on the type, and then the key itself must verify. That gives you two distinct failure modes, and IOS logs them with two different messages.

Failure mode one is a type mismatch. In the lab, R2's Ethernet0/1 was configured for message-digest authentication while R3 still had no authentication configured. With debug ip ospf adj running on R2, the reason appears immediately:

*Jul  4 23:33:57.737: OSPF-1 ADJ   Et0/1: Rcv pkt from 10.0.23.1 : Mismatched Authentication type. Input packet specified type 0, we use type 2

Read the line from right to left. "We use type 2" means the local interface expects MD5. "Input packet specified type 0" means the neighbor at 10.0.23.1 (R3) is sending unauthenticated packets. With the type table in mind (0 = null, 1 = simple password, 2 = cryptographic), this one debug line tells you which side is missing configuration without touching the other router.

Failure mode two is a key mismatch. Here the types agree, so the routers get one step further: both send type 2 packets, but the MD5 digest computed with the local key does not match the digest in the received packet. In the lab, message-digest authentication was enabled on both R2 and R3 but with different passwords:

*Jul  4 23:36:08.996: OSPF-1 ADJ   Et0/1: Rcv pkt from 10.0.23.1 : Mismatched Authentication key - ID 1

A key mismatch has two possible causes: the password strings differ, or the key IDs differ (the key ID is included in the digest calculation, so key 1 and key 2 with the same password still fail). Check both when you compare configurations.

Finding it

The symptom is a missing or bouncing neighbor. Because authentication is checked before any neighbor state machine processing, a new neighbor never even reaches INIT, so show ip ospf neighbor simply shows nothing on that interface (the state progression is covered in OSPF neighbor states explained). If the adjacency was already FULL when someone changed authentication on one side, the neighbor holds on until the dead timer expires, then drops. Either way, there is no error in the running config and nothing obviously broken, which is exactly why this failure feels invisible.

debug ip ospf adj is the tool that makes it visible. Both messages shown above come from the adjacency debug, not the hello debug, and the same debug also exposes area mismatches, so it is the first thing to run whenever a neighbor will not form (the wider checklist lives in troubleshooting OSPF neighbors not forming).

To see what authentication is active on your own interface without digging through the running config, filter show ip ospf interface:

R2# show ip ospf interface Ethernet0/1 | include authentication|key
  Cryptographic authentication enabled
    Youngest key id is 1

Run the same command on the neighbor and compare. If one side prints nothing about authentication and the other says cryptographic, you have found your type mismatch. If both say cryptographic, compare the key IDs, then reapply the key string on both sides (the string is stored encrypted, so retyping it is faster and more reliable than trying to decode what is there).

The fix

Make both ends identical: same authentication type, same key ID, same key string. For MD5 on the lab link, the interface configuration is:

R2(config)# interface Ethernet0/1
R2(config-if)# ip ospf authentication message-digest
R2(config-if)# ip ospf message-digest-key 1 md5 PingLabz123

Apply the same two commands (with the same key ID and password) on R3's Ethernet0/0. You can also enable authentication per area with area 1 authentication message-digest under router ospf 1, which turns it on for every interface in that area, but each interface still needs its own ip ospf message-digest-key. Mixing the two styles is a classic way to end up with a type mismatch on the one interface you forgot.

The success signal is the ADJCHG log message climbing to FULL. Seconds after the matching key was applied in the lab, R2 reported:

*Jul  4 23:36:57.301: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Ethernet0/1 from LOADING to FULL, Loading Done

And the neighbor table confirms a stable adjacency:

R2# show ip ospf neighbor Ethernet0/1
Neighbor ID     Pri   State           Dead Time   Address         Interface
3.3.3.3           0   FULL/  -        00:00:17    10.0.23.1       Ethernet0/1

The state reads FULL/ - (no DR role) because this link runs the point-to-point network type, so there is no DR election. On a broadcast segment you would see FULL/DR or FULL/BDR instead.

Rolling keys without an outage

That "Youngest key id is 1" line matters when you rotate passwords. IOS supports multiple message-digest keys on an interface, and during a rollover it sends a copy of each packet authenticated with every configured key, while accepting packets that match any of them. The youngest (most recently configured) key is the one the router prefers, and the output above is how you confirm which key that is.

The safe rollover procedure: add the new key with a higher key ID on both routers, verify the adjacency stays FULL and the youngest key ID has changed, then remove the old key from both sides with no ip ospf message-digest-key 1. At no point do the routers lose a shared valid key, so the adjacency never drops. If you are rolling out authentication for the first time rather than rotating keys, the full setup walkthrough is in the OSPF authentication configuration guide.

Key Takeaways

• Authentication failures come in two flavors: type mismatch ("Mismatched Authentication type", with type 0 = none, 1 = simple, 2 = cryptographic) and key mismatch ("Mismatched Authentication key - ID x").
• debug ip ospf adj exposes both instantly; the neighbor table alone just shows an absent or dying neighbor.
• Verify with show ip ospf interface | include authentication|key on both routers, and remember the key ID is part of the digest, so IDs must match too.
• The fix is symmetry: same type, same key ID, same string on both ends, confirmed by the ADJCHG message reaching FULL.
• Rotate keys by adding a new higher-numbered key on both sides before deleting the old one; the youngest key wins and the adjacency never drops.