An MPLS L3VPN route makes a long journey. It starts as an IP prefix in a customer's routing table, gets learned by a PE into a VRF, redistributed into MP-BGP, tagged with a route target, given a VPN label, advertised across an iBGP session, filtered on import at the far end, installed into another VRF, and finally handed to another customer router. There are at least six places it can quietly disappear, and none of them log an error.
This article gives you a repeatable path to walk. Follow the route from where it exists to where it does not, and the gap is the fault. Every capture is from a live five-router IOS XE lab where each failure was deliberately induced. For label-plane problems (LDP, LSPs) see Troubleshooting MPLS: LDP neighbors, label bindings, and broken LSPs, and the MPLS cluster guide for everything else.
The Path a Route Takes
Learn this sequence and you can troubleshoot any L3VPN. Each arrow is a place a route can vanish:
CE1 routing table
| (1) PE-CE protocol: static / OSPF / EIGRP / eBGP
PE1 VRF routing table <-- show ip route vrf CUST-A
| (2) redistribution into MP-BGP (NOT automatic, except with eBGP)
PE1 VPNv4 BGP table <-- show ip bgp vpnv4 vrf CUST-A
| (3) export: RT stamped on, VPN label allocated
| (4) MP-iBGP VPNv4 session (needs send-community extended)
PE2 VPNv4 BGP table <-- show ip bgp vpnv4 all
| (5) import: RT must match PE2's import list
PE2 VRF routing table <-- show ip route vrf CUST-A
| (6) PE-CE protocol back out
CE2 routing tableThe method: pick a prefix. Check for it at every stage, starting at the origin. The moment it is missing, you have found the broken link. Do not start in the middle, and do not start by pinging.
Failure 1: The Route Is in the VRF but Not in BGP
By far the most common L3VPN mistake, and it does exactly what you would fear: nothing. No log message, no error, no symptom on the local PE at all.
Here is PE2 in the lab. It has a static route for the customer's site-2 LAN, pointing at CE2, sitting happily in the VRF table:
PE2#show ip route vrf CUST-A static
Routing Table: CUST-A
10.0.0.0/8 is variably subnetted, 4 subnets, 3 masks
S 10.20.20.0/24 [1/0] via 10.20.2.2The route exists. It is correct. It even works locally: PE2 can ping into the customer site. Now look one layer up:
PE2#show ip bgp vpnv4 vrf CUST-A
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65000:1 (default for vrf CUST-A)
*>i 1.1.1.1/32 10.255.0.1 0 100 0 65001 i
*>i 10.20.1.0/30 10.255.0.1 0 100 0 ?
*> 10.20.2.0/30 0.0.0.0 0 32768 ?
*>i 192.168.99.0 10.255.0.1 0 100 0 65001 i10.20.20.0/24 is not there. The route is in the VRF routing table but was never handed to MP-BGP, so it was never advertised anywhere. The far end knows nothing about it:
PE1#show ip route vrf CUST-A 10.20.20.0
Routing Table: CUST-A
% Subnet not in table
PE1#ping vrf CUST-A 10.20.20.1 source Ethernet0/0
Sending 5, 100-byte ICMP Echos to 10.20.20.1, timeout is 2 seconds:
.....
Success rate is 0 percent (0/5)The cause is one missing line:
PE2#show run | section address-family ipv4 vrf CUST-A
address-family ipv4 vrf CUST-A
redistribute connectedredistribute connected is there. redistribute static is not. Static routes in a VRF do not reach MP-BGP on their own; you have to say so:
PE2(config)# router bgp 65000
PE2(config-router)# address-family ipv4 vrf CUST-A
PE2(config-router-af)# redistribute staticAnd the route appears on the far PE within seconds, complete with its VPN label:
PE1#show ip route vrf CUST-A 10.20.20.0
Routing entry for 10.20.20.0/24
Known via "bgp 65000", distance 200, metric 0, type internal
* 10.255.0.2 (default), from 10.255.0.2, 00:00:10 ago
MPLS label: 22
MPLS Flags: MPLS Required
PE1#ping vrf CUST-A 10.20.20.1 source Ethernet0/0
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/3/5 msThe same trap applies to OSPF (redistribute ospf 10) and EIGRP (redistribute eigrp 100, which additionally does nothing without a metric). Only eBGP as the PE-CE protocol is exempt, because the routes are already in BGP. The full comparison is in PE-CE routing protocols in MPLS L3VPN.
The tell: route present with show ip route vrf, absent from show ip bgp vpnv4 vrf, on the same router. Nothing else needs checking.
Failure 2: The Route Is Advertised but Never Imported
The route makes it into MP-BGP. It crosses the core. It arrives at the far PE. And it is thrown away at the door, because the receiving VRF is not importing its route target.
Remove one line from PE1:
PE1(config)# vrf definition CUST-A
PE1(config-vrf)# address-family ipv4
PE1(config-vrf-af)# no route-target import 65000:1The BGP session is untouched. The RD is unchanged. The export is still there. Now:
PE1#show ip bgp vpnv4 all summary
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.20.1.2 4 65001 6 12 43 0 0 00:01:33 2
10.255.0.2 4 65000 24 19 43 0 0 00:07:35 1The VPNv4 session to PE2 is up and has been for over seven minutes. But PfxRcd is 1, down from 4. The RT filter is discarding routes as they arrive. Chase the prefix:
PE1#show ip bgp vpnv4 rd 65000:1 10.20.20.0/24
BGP routing table entry for 65000:1:10.20.20.0/24, version 42
Paths: (0 available, no best path)
Not advertised to any peerPaths: (0 available, no best path) is the signature. BGP has an entry for the prefix, so it clearly heard about it, but there is no usable path, because no VRF has claimed it. And the confirming command:
PE1#show vrf detail CUST-A | include Export|Import|VRF
VRF CUST-A (VRF Id = 1); default RD 65000:1; default VPNID <not set>
Export VPN route-target communities
No Import VPN route-target communities
VRF label distribution protocol: not configuredNo Import VPN route-target communities. There is your fault, stated in plain English.
Meanwhile the customer is looking at this:
j@llmbits:~$ ping -c 3 10.20.20.1
PING 10.20.20.1 (10.20.20.1) 56(84) bytes of data.
From 192.168.99.1 icmp_seq=1 Destination Host Unreachable
From 192.168.99.1 icmp_seq=2 Destination Host Unreachable
From 192.168.99.1 icmp_seq=3 Destination Host Unreachable
--- 10.20.20.1 ping statistics ---
3 packets transmitted, 0 received, +3 errors, 100% packet lossEvery core protocol is healthy. LDP is up, OSPF is up, MP-BGP is up. One import statement is missing. If the RD and RT distinction is still fuzzy, RD vs RT explained takes it apart properly.
The tell: PfxRcd lower than expected on a healthy session, plus 0 available, no best path for a prefix that clearly arrived.
Failure 3: The RTs Were Stripped in Transit
Same symptom, different cause, and this one is nastier because the import configuration is correct.
Route targets are BGP extended communities. Extended communities are only sent if the neighbor is configured to send them. Remove that from PE2's VPNv4 neighbor:
PE2(config)# router bgp 65000
PE2(config-router)# address-family vpnv4
PE2(config-router-af)# no neighbor 10.255.0.1 send-community extendedThe result on PE1:
PE1#show ip bgp vpnv4 all summary
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.20.1.2 4 65001 11 20 56 0 0 00:06:21 2
10.255.0.2 4 65000 44 25 56 0 0 00:12:24 0
PE1#show ip bgp vpnv4 rd 65000:1 10.20.20.0/24
BGP routing table entry for 65000:1:10.20.20.0/24, version 55
Paths: (0 available, no best path)
Not advertised to any peerZero prefixes received. Not one. The session is established, messages are flowing (44 received), PE2 is dutifully advertising all its VPNv4 routes, and PE1 accepts none of them, because every route arrives with no route target at all and therefore matches no import list anywhere on the box.
PfxRcd of exactly 0 on an established VPNv4 session, with the far side definitely advertising, means one thing: check send-community extended. On both ends.
The tell: PfxRcd = 0, session Established, no RTs visible on any received route.
Failure 4: The Route Is Perfect and the Traffic Still Drops
Everything above is control plane. This one is not. The route is in BGP, imported into the VRF, installed in the routing table, and the packets still go nowhere, because there is no label to put on them.
PE1#show ip route vrf CUST-A 10.20.20.0 255.255.255.0
Routing entry for 10.20.20.0/24
Known via "bgp 65000", distance 200, metric 0, type internal
* 10.255.0.2 (default), from 10.255.0.2
MPLS label: 22
MPLS Flags: MPLS Required
PE1#show ip cef vrf CUST-A 10.20.20.0
10.20.20.0/24
nexthop 10.30.30.2 Ethernet0/1 unusable: no labelThe routing table is fine. CEF, which is what actually forwards the packet, says unusable: no label. The BGP next hop (PE2's loopback) has no transport LSP, because LDP is broken somewhere in the core. The VPN label 22 is known; the label to get to PE2 is not, and you cannot build the stack with only the inner label.
This is where the two troubleshooting articles meet. If your route is present and correct in the VRF table but traffic drops, stop looking at BGP and go to the LDP article. The diagnostic sequence there (OSPF FULL, loopback ping succeeds, show mpls ldp neighbor empty) takes about ninety seconds.
The tell: route present in the VRF, show ip cef vrf says unusable: no label.
Failure 5: The Route Reaches the Far PE but Not the Far CE
The last hop. Everything in the provider network is correct, the route is sitting in PE2's CUST-A table, and the customer still cannot see it, because it was never handed back out to the CE.
Failure 1 in reverse: redistribution runs in both directions and people configure one. With OSPF as the PE-CE protocol you need two statements on each PE:
router ospf 10 vrf CUST-A
redistribute bgp 65000 subnets <-- VPN routes out to the CE
!
router bgp 65000
address-family ipv4 vrf CUST-A
redistribute ospf 10 <-- CE routes into the VPNConfigure only the second and traffic flows exactly one way, which produces the memorable symptom of a ping that fails while a ping in the opposite direction succeeds. Check the CE's table directly; it is the fastest test:
CE2#show ip route ospf
1.0.0.0/32 is subnetted, 1 subnets
O IA 1.1.1.1 [110/21] via 10.20.2.1, 00:00:37, Ethernet0/1
10.0.0.0/8 is variably subnetted, 5 subnets, 3 masks
O IA 10.20.1.0/30 [110/11] via 10.20.2.1, 00:00:37, Ethernet0/1
O IA 192.168.99.0/24 [110/30] via 10.20.2.1, 00:00:37, Ethernet0/1(The O IA is expected, not a fault. The MPLS backbone acts as an OSPF superbackbone, so VPN routes arrive as inter-area.)
With eBGP as the PE-CE protocol there is a different last-hop trap: if the customer uses the same AS number at every site, CE2 will reject routes originated by CE1 because it sees its own AS in the path. The fix is as-override on the PE, and the proof is in the AS path:
CE2#show ip bgp
Network Next Hop Metric LocPrf Weight Path
*> 192.168.99.0 10.20.2.1 0 65000 65000 iPath 65000 65000: the customer's 65001 has been overwritten by the provider's AS, so CE2 no longer sees itself in the path and accepts the route.
The Six Commands, In Order
show ip route vrf X (originating PE)Is the customer route even here? If not, the PE-CE protocol is the problem.show ip bgp vpnv4 vrf X (originating PE)In the VRF but not here? Missing redistribution.show ip bgp vpnv4 all summary (receiving PE)Session up? PfxRcd sensible? 0 means send-community extended.show ip bgp vpnv4 rd <rd> <prefix>"0 available, no best path" means it arrived and was rejected on import.show vrf detail XConfirms the import and export RT lists. Reads in plain English.show ip cef vrf X <prefix>Route perfect but traffic dropping? "unusable: no label" points at LDP.FAQ
The VPNv4 session is up but PfxRcd is 0. Where do I start?
send-community extended on the advertising PE. That is the cause more often than anything else. If it is present, confirm the far PE is actually exporting (check show ip bgp vpnv4 vrf X on the sender, not the receiver).
Why does the route show in BGP but not in the VRF routing table?
The RT on the route does not match any RT in the VRF's import list. Run show vrf detail on the receiving PE and compare against the extended community on the route.
The route is in the VRF table and I still cannot ping.
You have crossed from a control-plane problem to a data-plane one. Run show ip cef vrf X <prefix>. If it says unusable: no label, the transport LSP to the remote PE is broken and this is an LDP problem, not a VPN one.
Can I clear just the VPNv4 routes without bouncing the session?
Yes, use soft reconfiguration: clear ip bgp <neighbor> vpnv4 unicast soft in (or soft out on the advertising side). Route refresh is supported by every modern IOS XE image, so a hard clear is almost never necessary.
Key Takeaways
- Walk the route's path from origin to destination and stop at the first stage where it is missing. Do not start by pinging.
- A route in
show ip route vrfbut not inshow ip bgp vpnv4 vrfon the same router means redistribution is missing. This is the most common L3VPN fault. - PfxRcd lower than expected with a healthy session means a route-target import problem.
show vrf detailwill sayNo Import VPN route-target communitiesin so many words. - PfxRcd of exactly 0 on an established VPNv4 session means
send-community extendedis missing and the RTs are being stripped. Paths: (0 available, no best path)means the route arrived and was rejected on import, not that it never came.- Perfect route plus dropped traffic plus
unusable: no labelis a broken LSP, not a BGP problem. Go to LDP. - Redistribution runs both ways. One-way redistribution produces one-way traffic.
See also: Troubleshooting MPLS: LDP, label bindings and broken LSPs, the full L3VPN configuration walkthrough, and the MPLS cluster guide.
