MPLS L3VPN is the service that made MPLS a service-provider standard, and it is still what most enterprises are actually buying when their carrier sells them "an MPLS circuit." The concept is simple to state and surprisingly fiddly to configure: many customers, each with their own private (and possibly overlapping) IP addressing, carried across one shared provider backbone, with each customer seeing only their own routes.
This article is the configuration walkthrough. Every command and every piece of output below comes from a five-router Cisco IOS XE lab (CE1 - PE1 - P - PE2 - CE2) running OSPF as the core IGP, LDP for label distribution, and MP-iBGP carrying VPNv4 between the two PE routers. A real Debian host sits behind CE1 and pings across the VPN at the end. If you want the wider context first, start with the MPLS cluster guide.
The Four Moving Parts
Before touching a keyboard, get the pieces straight. An L3VPN is built from four things stacked on top of each other, and if you configure them out of order you will spend an hour chasing a symptom caused by a missing layer underneath.
Notice what is missing from that list: the P router in the middle of the core runs items 1 and 2 and nothing else. It has no VRFs, no BGP, and no idea that any customer exists. We will prove that with real output later, because it is the single most important architectural fact about L3VPN and the thing most people get wrong when they first draw the diagram.
The Lab
Five IOS XE routers in a line, plus a real Linux host bridged in behind CE1 so the final ping is not simulated:
VM 192.168.99.100
|
[ CE1 ] --- [ PE1 ] --- [ P ] --- [ PE2 ] --- [ CE2 ]
10.20.1.2/30 .1/30 10.20.2.2/30
10.30.30.1/24 .2 .1 10.30.31.2/24
Loopbacks: PE1 10.255.0.1 P 10.255.0.3 PE2 10.255.0.2
Customer site 1 LAN: 192.168.99.0/24 (the real VM)
Customer site 2 LAN: 10.20.20.0/24 (CE2 Loopback1)Provider AS is 65000. Customer VRF is CUST-A with route distinguisher 65000:1 and route target 65000:1 imported and exported. A second VRF, CUST-B, gets RD/RT 65000:2 and deliberately overlapping addressing, which we use at the end to prove the separation is real.
Step 1: Core IGP and Loopbacks
Everything hangs off the PE loopbacks, so configure them first and get them into the IGP. The loopback is the LDP router ID, the BGP update source, and the BGP next hop that the far-end PE resolves against. If the loopback is not in the IGP with a /32, nothing else will work.
PE1(config)# interface Loopback0
PE1(config-if)# ip address 10.255.0.1 255.255.255.255
PE1(config-if)# interface Ethernet0/1
PE1(config-if)# description Core link to P
PE1(config-if)# ip address 10.30.30.1 255.255.255.0
PE1(config-if)# router ospf 1
PE1(config-router)# router-id 10.255.0.1
PE1(config-router)# network 10.255.0.1 0.0.0.0 area 0
PE1(config-router)# network 10.30.30.0 0.0.0.255 area 0P and PE2 get the same treatment. The P router carries both core links:
P(config)# router ospf 1
P(config-router)# router-id 10.255.0.3
P(config-router)# network 10.255.0.3 0.0.0.0 area 0
P(config-router)# network 10.30.30.0 0.0.0.255 area 0
P(config-router)# network 10.30.31.0 0.0.0.255 area 0Verify before moving on. On a broadcast Ethernet segment the adjacency sits at 2WAY/DROTHER for a few tens of seconds during DR election, so do not panic if the first look is not FULL:
PE1#show ip ospf neighbor
Neighbor ID Pri State Dead Time Address Interface
10.255.0.3 1 FULL/DR 00:00:33 10.30.30.2 Ethernet0/1If you need the OSPF side of this in more depth, the OSPF cluster guide covers adjacency states and area design.
Step 2: LDP Across the Core
The IGP gives you IP reachability. LDP turns that into a label switched path. Two lines globally, one line per core-facing interface:
PE1(config)# mpls label protocol ldp
PE1(config)# mpls ldp router-id Loopback0 force
PE1(config)# interface Ethernet0/1
PE1(config-if)# mpls ipThe mpls ldp router-id Loopback0 force line is not optional in practice. Without it, LDP picks the highest interface address as its ID, and when that interface goes down the LDP session resets. Pin it to the loopback that the IGP already advertises.
mpls ip goes only on core-facing interfaces. Never on the interface facing a CE router, because the customer does not speak MPLS. Verify:
PE1#show mpls ldp neighbor
Peer LDP Ident: 10.255.0.3:0; Local LDP Ident 10.255.0.1:0
TCP connection: 10.255.0.3.60695 - 10.255.0.1.646
State: Oper; Msgs sent/rcvd: 8/8; Downstream
Up time: 00:00:34
LDP discovery sources:
Ethernet0/1, Src IP addr: 10.30.30.2
Addresses bound to peer LDP Ident:
10.255.0.3 10.30.30.2 10.30.31.1
PE1#show mpls interfaces
Interface IP Tunnel BGP Static Operational
Ethernet0/1 Yes (ldp) No No No YesState: Oper is what you want. Anything else and the LSP is not built. The LDP deep-dive covers discovery, session establishment, and label advertisement modes.
Once LDP is up, the P router has already built the transport LSPs to both PE loopbacks:
P#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 10.255.0.1/32 4146 Et0/0 10.30.30.1
17 Pop Label 10.255.0.2/32 3665 Et0/1 10.30.31.2"Pop Label" is penultimate hop popping. P is one hop away from each PE, so instead of swapping the transport label it strips it and hands the egress PE a packet with only the VPN label left. That saves the egress PE a lookup and is the default behaviour, not something you configure.
Step 3: Define the VRF, the RD, and the RTs
Now the customer. On both PEs:
PE1(config)# vrf definition CUST-A
PE1(config-vrf)# rd 65000:1
PE1(config-vrf)# address-family ipv4
PE1(config-vrf-af)# route-target export 65000:1
PE1(config-vrf-af)# route-target import 65000:1
PE1(config-vrf-af)# exit-address-familyThree values, three different jobs, and confusing them is the number-one L3VPN mistake:
If that distinction is still fuzzy, read RD vs RT: the most confused concepts in MPLS L3VPN, which proves the difference with two overlapping prefixes in one BGP table.
Now put the customer-facing interface into the VRF. Do this before configuring the IP address, because moving an interface into a VRF wipes its IP configuration:
PE1(config)# interface Ethernet0/0
PE1(config-if)# description To CE1 - VRF CUST-A
PE1(config-if)# vrf forwarding CUST-A
PE1(config-if)# ip address 10.20.1.1 255.255.255.252Check it landed:
PE1#show vrf
Name Default RD Protocols Interfaces
CUST-A 65000:1 ipv4 Et0/0
CUST-B 65000:2 ipv4 Lo11This is exactly the same VRF construct used in VRF-Lite. The difference is that VRF-Lite stops here and hand-carries the separation hop by hop, while L3VPN hands the VRF's routes to MP-BGP and lets the backbone do the work.
Step 4: MP-BGP VPNv4 Between the PEs
The PEs peer with each other over their loopbacks, inside the provider AS, and activate the VPNv4 address family:
PE1(config)# router bgp 65000
PE1(config-router)# no bgp default ipv4-unicast
PE1(config-router)# neighbor 10.255.0.2 remote-as 65000
PE1(config-router)# neighbor 10.255.0.2 update-source Loopback0
PE1(config-router)# address-family vpnv4
PE1(config-router-af)# neighbor 10.255.0.2 activate
PE1(config-router-af)# neighbor 10.255.0.2 send-community extended
PE1(config-router-af)# exit-address-familyFour lines matter here and every one of them is a classic outage:
The MP-BGP details, including how the VPNv4 address family relates to the IPv4 and IPv6 ones, are covered in MP-BGP: Multiprotocol BGP address families explained. For BGP session mechanics generally, see the BGP cluster guide.
Confirm the session:
PE1#show ip bgp vpnv4 all summary
BGP router identifier 10.255.0.1, local AS number 65000
Neighbor V AS MsgRcvd MsgSent TblVer InQ OutQ Up/Down State/PfxRcd
10.255.0.2 4 65000 19 17 40 0 0 00:06:40 4Step 5: PE-CE Routing
The PE now needs to learn the customer's prefixes and give the customer the far-end's. You have four options: static, OSPF, EIGRP, or eBGP. All four are covered in detail with lab output in PE-CE routing protocols in MPLS L3VPN. Here is the simplest one to get you to a working VPN, static:
PE1(config)# ip route vrf CUST-A 192.168.99.0 255.255.255.0 10.20.1.2
PE1(config)# ip route vrf CUST-A 1.1.1.1 255.255.255.255 10.20.1.2
PE1(config)# router bgp 65000
PE1(config-router)# address-family ipv4 vrf CUST-A
PE1(config-router-af)# redistribute connected
PE1(config-router-af)# redistribute staticTwo things are happening. The ip route vrf lines put the customer's prefixes into the CUST-A table. The redistribute lines under the VRF address family push them into MP-BGP, where they become VPNv4 routes tagged with RT 65000:1 and given a VPN label.
Forgetting the redistribution is the classic "I can see the route on the PE but the far side never gets it" failure, and it is walked through step by step in Troubleshooting MPLS L3VPN.
The CE side is boringly simple. The CE is not VRF-aware and does not run MPLS; as far as it knows, PE1 is just a router:
CE1(config)# ip route 0.0.0.0 0.0.0.0 10.20.1.1Verifying the VPN
Start at the VRF routing table on PE1. The customer's local routes are connected or static; the far-end site arrives as BGP:
PE1#show ip route vrf CUST-A
Routing Table: CUST-A
1.0.0.0/32 is subnetted, 1 subnets
S 1.1.1.1 [1/0] via 10.20.1.2
2.0.0.0/32 is subnetted, 1 subnets
B 2.2.2.2 [200/0] via 10.255.0.2, 00:00:11
10.0.0.0/8 is variably subnetted, 4 subnets, 3 masks
C 10.20.1.0/30 is directly connected, Ethernet0/0
L 10.20.1.1/32 is directly connected, Ethernet0/0
B 10.20.2.0/30 [200/0] via 10.255.0.2, 00:00:11
B 10.20.20.0/24 [200/0] via 10.255.0.2, 00:00:11
S 192.168.99.0/24 [1/0] via 10.20.1.2Note the next hop on the BGP routes: 10.255.0.2, PE2's loopback. Not a physical interface, not a P router. The VRF's forwarding decision is "hand this to PE2 over the LSP."
Now the VPNv4 table, which is where the RD and the VPN label live:
PE1#show ip bgp vpnv4 vrf CUST-A 10.20.20.0
BGP routing table entry for 65000:1:10.20.20.0/24, version 13
Paths: (1 available, best #1, table CUST-A)
Refresh Epoch 1
Local
10.255.0.2 (metric 21) (via default) from 10.255.0.2 (10.255.0.2)
Origin incomplete, metric 0, localpref 100, valid, internal, best
Extended Community: RT:65000:1
mpls labels in/out nolabel/22Read that carefully, because it contains the whole protocol. The prefix is 65000:1:10.20.20.0/24, RD glued to the front. The extended community is RT:65000:1, which is why the CUST-A VRF imported it. And the outgoing MPLS label is 22, the VPN label PE2 allocated for this prefix. PE1 will push 22 as the inner label on every packet destined for that subnet.
The label forwarding state on PE1:
PE1#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 10.255.0.3/32 0 Et0/1 10.30.30.2
17 Pop Label 10.30.31.0/24 0 Et0/1 10.30.30.2
18 17 10.255.0.2/32 0 Et0/1 10.30.30.2
19 No Label 10.20.20.0/24[V] 500 aggregate/CUST-B
20 No Label 10.20.1.0/30[V] 1140 aggregate/CUST-A
21 No Label 1.1.1.1/32[V] 0 Et0/0 10.20.1.2
22 No Label 192.168.99.0/24[V] \
570 Et0/0 10.20.1.2Entry 18 is the transport LSP toward PE2: swap local label 18 for outgoing label 17 and send it to P. The [V] entries are the VPN labels PE1 has handed out for its own customer prefixes, telling PE2 which VRF and which next hop to use for traffic coming back the other way. The label format article breaks down the two-label stack byte by byte.
The P Router Is BGP-Free (and That Is the Whole Point)
Here is the capture that should reframe how you think about L3VPN. This is the P router, sitting in the middle of the path, forwarding customer traffic all day:
P#show ip bgp summary
% BGP not active
P#show ip route 10.20.20.0
% Subnet not in table
P#show mpls forwarding-table
Local Outgoing Prefix Bytes Label Outgoing Next Hop
Label Label or Tunnel Id Switched interface
16 Pop Label 10.255.0.1/32 4146 Et0/0 10.30.30.1
17 Pop Label 10.255.0.2/32 3665 Et0/1 10.30.31.2The P router does not run BGP. It has never heard of 10.20.20.0/24. It has no VRFs. Its entire forwarding table is two labels pointing at two PE loopbacks. Customer traffic crosses it as a labelled packet and it swaps or pops the outer label without ever looking at the IP header.
That is why L3VPN scales. Add a thousand customers with a million routes and the P routers do not grow at all; only the PEs carry VPN state. It is also why "the core is BGP-free" is a design goal you will hear repeated in every service provider architecture conversation.
End to End: A Real Host Across the VPN
A real Debian machine sits on the 192.168.99.0/24 LAN behind CE1. It has no idea MPLS exists. It pings the far site:
j@llmbits:~$ ping -c 4 10.20.20.1
PING 10.20.20.1 (10.20.20.1) 56(84) bytes of data.
64 bytes from 10.20.20.1: icmp_seq=1 ttl=251 time=7.16 ms
64 bytes from 10.20.20.1: icmp_seq=2 ttl=251 time=7.02 ms
64 bytes from 10.20.20.1: icmp_seq=3 ttl=251 time=7.07 ms
64 bytes from 10.20.20.1: icmp_seq=4 ttl=251 time=6.99 ms
--- 10.20.20.1 ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3004msAnd the traceroute from CE1, which is where the label stack finally becomes visible:
CE1#traceroute 10.20.20.1 source Ethernet0/0
Type escape sequence to abort.
Tracing the route to 10.20.20.1
1 10.20.1.1 2 msec
2 10.30.30.2 [MPLS: Labels 17/22 Exp 0] 5 msec
3 10.20.2.1 [MPLS: Label 22 Exp 0] 5 msec
4 10.20.2.2 6 msecHop 2 is the P router, and the packet arrives carrying two labels: 17 (transport, "get me to PE2") and 22 (VPN, "this belongs to CUST-A and goes out toward CE2"). Hop 3 is PE2, and the packet now has only one label, because P performed penultimate hop popping and stripped the transport label. PE2 pops the VPN label, does an IP lookup in the CUST-A table, and forwards to CE2.
That single traceroute is the entire L3VPN forwarding plane on one screen.
Proving the Separation: A Second VRF with Overlapping Addressing
The claim is that two customers can use the same IP space. Let us actually test it rather than take it on faith. VRF CUST-B gets RD/RT 65000:2, and on PE1 it is given a network that deliberately collides with CUST-A's site-2 LAN:
PE1(config)# vrf definition CUST-B
PE1(config-vrf)# rd 65000:2
PE1(config-vrf)# address-family ipv4
PE1(config-vrf-af)# route-target export 65000:2
PE1(config-vrf-af)# route-target import 65000:2
PE1(config)# interface Loopback11
PE1(config-if)# vrf forwarding CUST-B
PE1(config-if)# ip address 10.20.20.1 255.255.255.010.20.20.0/24 now exists twice on the same router. Here is the same prefix in the same BGP table, twice, under two different RDs:
PE1#show ip bgp vpnv4 all
Network Next Hop Metric LocPrf Weight Path
Route Distinguisher: 65000:1 (default for vrf CUST-A)
*> 1.1.1.1/32 10.20.1.2 0 0 65001 i
*>i 2.2.2.2/32 10.255.0.2 0 100 0 65001 i
*> 10.20.1.0/30 0.0.0.0 0 32768 ?
*>i 10.20.2.0/30 10.255.0.2 0 100 0 ?
*>i 10.20.20.0/24 10.255.0.2 0 100 0 65001 i
*> 192.168.99.0 10.20.1.2 0 0 65001 i
Route Distinguisher: 65000:2 (default for vrf CUST-B)
*> 10.20.20.0/24 0.0.0.0 0 32768 ?
*>i 10.20.99.0/24 10.255.0.2 0 100 0 ?Both entries for 10.20.20.0/24 coexist because to BGP they are different routes: 65000:1:10.20.20.0/24 and 65000:2:10.20.20.0/24. That is the entire job of the route distinguisher.
And the forwarding follows:
PE1#show ip route vrf CUST-A 10.20.20.0 255.255.255.0
Routing entry for 10.20.20.0/24
Known via "bgp 65000", distance 200, metric 0, type internal
* 10.255.0.2 (default), from 10.255.0.2
MPLS label: 22
MPLS Flags: MPLS Required
PE1#show ip route vrf CUST-B 10.20.20.0 255.255.255.0
Routing entry for 10.20.20.0/24
Known via "connected", distance 0, metric 0 (connected, via interface)
* directly connected, via Loopback11Same prefix, same router, two completely different forwarding decisions. In CUST-A it goes across the MPLS core to PE2 with VPN label 22. In CUST-B it is a local interface. Neither customer can see the other's route, and no configuration effort was required to keep them apart beyond giving them different RDs and RTs.
The Complete PE Configuration
Everything above, on PE1, condensed:
vrf definition CUST-A
rd 65000:1
address-family ipv4
route-target export 65000:1
route-target import 65000:1
!
mpls label protocol ldp
mpls ldp router-id Loopback0 force
!
interface Loopback0
ip address 10.255.0.1 255.255.255.255
!
interface Ethernet0/0
vrf forwarding CUST-A
ip address 10.20.1.1 255.255.255.252
!
interface Ethernet0/1
ip address 10.30.30.1 255.255.255.0
mpls ip
!
router ospf 1
router-id 10.255.0.1
network 10.255.0.1 0.0.0.0 area 0
network 10.30.30.0 0.0.0.255 area 0
!
router bgp 65000
no bgp default ipv4-unicast
neighbor 10.255.0.2 remote-as 65000
neighbor 10.255.0.2 update-source Loopback0
address-family vpnv4
neighbor 10.255.0.2 activate
neighbor 10.255.0.2 send-community extended
exit-address-family
address-family ipv4 vrf CUST-A
redistribute connected
redistribute static
exit-address-familyThe P router config is a quarter of that length and contains no customer state at all. That asymmetry is the design.
FAQ
Do I need MPLS on the PE-CE link?
No, and you should not put it there. The CE is a plain IP router. mpls ip belongs only on core-facing interfaces between PE and P routers.
Can the RD and RT be the same value?
Yes, and in a simple any-to-any VPN they usually are (65000:1 for both, as in this lab). They are still doing completely different jobs. In hub-and-spoke or extranet designs they diverge immediately.
Why iBGP between the PEs and not eBGP?
Both PEs are inside the same provider AS, so the VPNv4 session is iBGP. In a real network you would not full-mesh every PE; you would point them all at a pair of route reflectors configured for the VPNv4 address family.
What happens if the core IGP is fine but LDP is down?
The VPN dies while every IP-level test passes. It is the single most confusing MPLS failure mode, and it has its own article: Troubleshooting MPLS: LDP neighbors, label bindings, and broken LSPs.
Key Takeaways
- Build the layers in order: IGP, then LDP, then VRF/RD/RT, then MP-BGP VPNv4, then PE-CE routing. Verify each before moving up.
- The PE loopback is load-bearing. It is the LDP router ID, the BGP update source, and the next hop that must resolve to a label.
- RD makes prefixes unique; RT decides who imports them. Same syntax, unrelated jobs.
send-community extendedis mandatory. Without it the RTs are stripped and every VPN route is silently discarded on import.- Customer prefixes only reach MP-BGP if you redistribute them under
address-family ipv4 vrf. This is the most common "route is on the PE but not in BGP" cause. - The P router runs no BGP and holds no customer routes. Two labels in its forwarding table carry every customer.
- A labelled traceroute (
Labels 17/22thenLabel 22) shows the transport and VPN labels, and PHP, in one command.
Next in the cluster: the four PE-CE routing options compared, and the MPLS cluster guide for everything else.
