OSPF has an unhelpful habit: when you break it, it usually says nothing. An adjacency that should form simply does not. A route that should be in the table simply is not. There is one failure mode - exactly one - where OSPF actually tells you what is wrong, and we cover it below so you can enjoy the novelty.
The rest of the time, diagnosis is a matter of knowing which two commands to compare. This article is five OSPF faults, each built and broken for real in a CML lab, with the actual output and the actual fix. For the theory, see the complete OSPF guide.
The method
Every OSPF problem is one of three things, in this order:
show ip ospf neighbor. Not FULL (or 2WAY on a DROTHER pair) means stop here. Nothing else matters.
show ip ospf database. Is the LSA there? If not, it was never generated or was filtered at an ABR/ASBR.
The single highest-value habit: when an adjacency will not form, run show ip ospf interface <int> on both ends and diff them. Nine of the ten reasons an OSPF adjacency fails are visible in that one output, side by side.
show ip ospf interface Ethernet0/1 | include Network Type|Timer intervals|Area|Cost|AuthenticationThe five things that must match for two OSPF routers to become neighbours: area ID, hello interval, dead interval, authentication, and stub/NSSA area flags. Subnet mask must also match. MTU must match for the adjacency to progress past EXSTART.
Ticket 1: "The NSSA router lost its adjacency and nothing is in the log"
Symptom. R5 (inside area 2) and R4 (its ABR) were FULL. After a change, R5 has vanished from R4's neighbour table. The interfaces are up. Pings between the two interface addresses work.
R4#show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
2.2.2.2 1 FULL/BDR 00:00:34 10.0.24.1 Ethernet0/2
1.1.1.1 1 FULL/BDR 00:00:37 10.0.14.1 Ethernet0/05.5.5.5 is simply not there. And the log:
R4#show logging | include ADJCHG
*Jul 12 07:02:46.213: %OSPF-5-ADJCHG: Process 1, Nbr 5.5.5.5 on Ethernet0/1 from FULL to DOWN,
Neighbor Down: Dead timer expired"Dead timer expired" is OSPF's way of saying "I stopped receiving valid Hellos and I am not going to tell you why". It is the least useful message in the protocol, and it is the one you will see most.
Diagnosis. Compare both ends:
R5#show ip ospf | include Area 2|It is a
Area 2
It is a stub area
R4#show ip ospf | include Area 2|It is a
Area 2
It is a NSSA areaCause. R5 was configured with area 2 stub while R4 has area 2 nssa. These set different bits in the Hello packet's Options field - the E-bit for stub, the N-bit for NSSA - and a router discards any Hello whose area flags do not match its own. The Hellos are being sent and received. They are being silently thrown away.
Fix: make the area type identical on every router in the area. This is not negotiable and there is no partial credit.
R5(config-router)#no area 2 stub
R5(config-router)#area 2 nssaLesson: area type is one of the five things that must match. It is invisible in show ip ospf neighbor and it produces no dedicated error. The tell is that everything else looks perfect.
Ticket 2: "We summarised at the ABR and now a whole subnet is black-holed"
Symptom. After an ABR summarisation change, hosts in one range are unreachable - but instead of the traffic being dropped near the source, it is being pulled across the network and dropped at the ABR.
R2(config-router)#area 1 range 10.0.224.0 255.255.224.0Diagnosis.
R1#show ip route ospf | include 10.0.224|10.0.234
O IA 10.0.224.0/19 [110/20] via 10.0.12.2, Ethernet0/1The specific 10.0.234.0/24 has been replaced by a /19 that covers a great deal of address space that does not exist. And on the ABR:
R2#show ip route 10.0.224.0 255.255.224.0
Routing entry for 10.0.224.0/19
Known via "ospf 1", distance 110, metric 10, type intra area
Routing Descriptor Blocks:
* directly connected, via Null0The data plane:
R1#ping 10.0.230.5 source Loopback0 repeat 2
U.
Success rate is 0 percent (0/2)U means an ICMP unreachable came back - from R2, which pulled the traffic in on the strength of its /19 advertisement and then dropped it on Null0.
Cause. The summary range is far wider than the address space that actually exists behind it. The Null0 discard route is correct anti-loop behaviour (without it the ABR would follow its own default route and loop), but it means a sloppy range actively attracts and destroys traffic for addresses that would otherwise have failed cleanly at the source.
Fix: tighten the range to exactly the address space that exists. area 1 range 10.0.234.0 255.255.255.0.
Lesson: a summary is a promise. The ABR is telling the whole network "I can reach everything in this /19". Only make promises you can keep. Full detail on all four filtering mechanisms in OSPF filtering compared.
Ticket 3: "Two adjacencies dropped at once" (the one OSPF actually tells you about)
Symptom. R2 lost both of its adjacencies to R3 simultaneously - across two completely different links.
R2#show logging | include DUP_RTRID|ADJCHG
*Jul 12 07:04:06.723: %OSPF-4-DUP_RTRID_NBR: OSPF detected duplicate router-id 2.2.2.2
from 10.0.23.2 on interface Ethernet0/1
*Jul 12 07:04:46.690: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Ethernet0/1 from FULL to DOWN,
Neighbor Down: Dead timer expired
*Jul 12 07:04:46.690: %OSPF-5-ADJCHG: Process 1, Nbr 3.3.3.3 on Ethernet0/2 from FULL to DOWN,
Neighbor Down: Dead timer expiredCause. Somebody set R3's router-id to 2.2.2.2, which is already R2's. OSPF's entire LSDB is keyed on router-id. Two routers claiming the same ID means their Router LSAs overwrite each other, the database becomes incoherent, and OSPF refuses to build an adjacency with an impostor.
Fix: unique router-ids everywhere. Set them explicitly with router-id x.x.x.x - never let OSPF pick one for you from whatever the highest loopback happens to be that day.
R3(config-router)#router-id 3.3.3.3
R3#clear ip ospf processLesson: enjoy it. %OSPF-4-DUP_RTRID_NBR is the one time OSPF names your mistake in plain language. The tell that should send you looking for it: multiple adjacencies to the same neighbour dropping at once, across unrelated links. Nothing physical does that.
Ticket 4: "Routers in the NSSA can't reach anything outside OSPF"
Symptom. R5, an internal router in NSSA area 2, has full reachability inside OSPF but cannot reach any redistributed external network.
R5#show ip route | include Gateway of last
Gateway of last resort is not set
R5#ping 172.20.20.1 source Loopback0
.....
Success rate is 0 percent (0/5)Diagnosis. An NSSA, by definition, blocks Type-5 external LSAs. That is what makes it a stub-family area. R5 therefore has no route to 172.20.20.0/24, and - here is the part people get wrong - an NSSA does not automatically get a default route either.
A regular stub area gets a default injected by the ABR automatically. A totally stubby area gets one automatically. An NSSA does not. You have to ask for it.
Fix, on the NSSA ABR:
R4(config-router)#area 2 nssa default-information-originateR5#show ip route 0.0.0.0
Routing entry for 0.0.0.0/0, supernet
Known via "ospf 1", distance 110, metric 1, candidate default path,
type NSSA extern 2, forward metric 10
* 10.0.45.1, from 4.4.4.4, via Ethernet0/0
R5#ping 172.20.20.1 source Loopback0
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 5/5/6 msNote the route type: NSSA extern 2. The default arrives as a Type-7 LSA, not a Type-3, which is consistent with the NSSA's whole design (Type-7 is the only external-carrying LSA an NSSA permits).
Lesson: know which area types auto-inject a default and which do not. Stub: yes. Totally stubby: yes. NSSA: no. Totally stubby NSSA: yes. It is arbitrary and you simply have to remember it.
Ticket 5: "The adjacency won't form and both configs look right"
Symptom. R1 and R2 are directly connected, interfaces up, pings work, both have the link in the correct area. No adjacency.
Diagnosis. Diff the two interfaces:
R1#show ip ospf interface Ethernet0/1 | include Network Type|Timer intervals
Process ID 1, Router ID 1.1.1.1, Network Type NON_BROADCAST, Cost: 10
Timer intervals configured, Hello 30, Dead 120, Wait 120, Retransmit 5
R2#show ip ospf interface Ethernet0/0 | include Network Type|Timer intervals
Process ID 2, Router ID 2.2.2.2, Network Type BROADCAST, Cost: 10
Timer intervals configured, Hello 10, Dead 40, Wait 40, Retransmit 5R2#show logging | include ADJCHG
*Jul 12 07:00:14.361: %OSPF-5-ADJCHG: Process 1, Nbr 1.1.1.1 on Ethernet0/0 from FULL to DOWN,
Neighbor Down: Dead timer expiredCause. Someone set ip ospf network non-broadcast on one side. Here is the mechanism, and it is subtle:
The network type is not carried in the Hello packet. OSPF has no way to detect a network type mismatch directly. But the network type determines the default timers, and the timers are in the Hello. Broadcast and point-to-point default to hello 10 / dead 40. Non-broadcast and point-to-multipoint default to hello 30 / dead 120. A router discards any Hello whose timers do not match its own. So the mismatch is detected, but as a timer mismatch, and reported as a dead-timer expiry.
The nasty variant: point-to-point and broadcast have the same default timers. So if you mismatch those two, the adjacency comes up - and then the two routers generate incompatible LSAs (one expects a Network LSA and a DR; the other does not), and routing breaks in a way that is far harder to diagnose than a down adjacency. Always check the network type even when the neighbour is FULL.
Fix: make the network type identical on both ends.
R1(config-if)#no ip ospf network non-broadcastBonus: the 2WAY/DROTHER pause everyone panics about
On a freshly-booted broadcast segment you will routinely see this for the first 30 to 40 seconds:
R4#show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
2.2.2.2 1 2WAY/DROTHER 00:00:39 10.0.24.1 Ethernet0/2
1.1.1.1 1 2WAY/DROTHER 00:00:39 10.0.14.1 Ethernet0/0This is not broken. It is the DR election wait timer (equal to the dead interval, 40 seconds by default) running its course. Wait for it. Two DROTHERs on a segment will also legitimately stay at 2WAY forever - they only form full adjacencies with the DR and BDR, never with each other. That is correct behaviour and not a fault.
The commands, collected
! Adjacency
show ip ospf neighbor
show ip ospf interface <int> | include Network Type|Timer intervals|Area|Cost|Authentication
show ip ospf | include Area|It is a|Number of areas
show logging | include OSPF|ADJCHG|DUP_RTRID
! Database
show ip ospf database
show ip ospf database router <rid>
show ip ospf database summary <prefix>
show ip ospf database external <prefix>
show ip ospf database external | include Forward Address <-- run this before you filter anything
show ip ospf database nssa-external <prefix>
! Routing table
show ip route <prefix> (look at: type, metric, forward metric)
show ip route ospf
show ip cef <prefix>
! Performance
show ip ospf statistics (SPF runs, timings, and the reason codes)Try it yourself
Build a multi-area topology with an NSSA, then break it. Ten minutes per fault, and name the cause from show output before you look at the config:
- Set
area X stubon one router in an NSSA. - Summarise at an ABR with a range far wider than the real address space, then ping a non-existent host inside it.
- Give two routers the same router-id.
- Build an NSSA and try to reach an external network from inside it.
- Set
ip ospf network non-broadcaston one end of an Ethernet link. Then trypoint-to-pointon one end instead, and note that the adjacency comes up and routing still breaks. - Bonus: enable
prefix-suppressionin a topology that has an external route with a non-zero forwarding address, and watch the external route vanish from the RIB while its LSA stays in the database. That one is covered in the OSPF forwarding address.
Key takeaways
- Work the layers in order: adjacency, database, routing table.
- Area ID, hello interval, dead interval, authentication, area type flags and subnet mask must all match. MTU must match to get past EXSTART.
show ip ospf interfaceon both ends, side by side, is the single most valuable diagnostic in OSPF.- Area-type mismatch and network-type mismatch both fail silently, reported only as a dead-timer expiry.
- Duplicate router-id is the one failure OSPF names in a log. Multiple adjacencies to one neighbour dropping at once is the signature.
- An NSSA does not get an automatic default route. Stub and totally-stubby areas do.
- A summary range that is wider than reality black-holes traffic at the summarising router, via a Null0 discard route that is doing exactly what it should.
- 2WAY/DROTHER for 40 seconds after boot is a DR election, not a fault.
That closes the expert OSPF series. The full cluster index, from first principles to here, lives on the OSPF pillar guide.
