OMP and BFD talk about colors and TLOCs. None of that starts until the WAN edge can reach its next hop in VPN 0, the transport VPN. That next hop is usually an MPLS provider edge (PE) or an internet CE. Most branches give the edge a static default toward each PE. Some sites speak BGP to the PE instead, the way a classic CE would.
This post looks at that underlay hop on Cisco Catalyst SD-WAN 20.18.2.1 with C8000v routers on IOS XE 17.18.02, built in my home lab. The edges already use two static defaults, one per color. Control connections and BFD are up through those next hops. I did not add transport-side BGP. show ip bgp summary still says BGP not active, and the PE-facing next hops in this lab do not speak BGP.
This is part of the Cisco SD-WAN series. Previous: VRRP at dual-edge SD-WAN sites. Next: Multi-Region Fabric basics.
For SD-WAN Manager (formerly vManage) and the other old names, see the name map.
The problem: the overlay sits on an underlay default
A WAN edge has to resolve three very ordinary questions before it can join the fabric:
- How do I reach the Validator, the Controller, and the Manager?
- How do I reach the far TLOC addresses on each color?
- If a circuit dies, do I still have a path on the other color?
VPN 0 holds the transport interfaces and those answers. Service VPNs never see the PE. If VPN 0 cannot ping the next hop, you will not get a DTLS session to debug.
Static routing is the common answer: one default (or a handful of specifics) out each WAN port. BGP is the answer when the PE already expects a CE session, or when you need to learn more than a default from the provider.
What this lab already proves with static defaults
edge1 uses GigabitEthernet2 for mpls at 10.10.1.11/24 and GigabitEthernet3 for biz-internet at 10.20.1.11/24. Two equal-cost static defaults point at the PE-facing addresses:
ip route 0.0.0.0 0.0.0.0 10.10.1.1
ip route 0.0.0.0 0.0.0.0 10.20.1.1The global table shows both:
edge1#show ip route
Gateway of last resort is 10.20.1.1 to network 0.0.0.0
S* 0.0.0.0/0 [1/0] via 10.20.1.1
[1/0] via 10.10.1.1
C 10.10.1.0/24 is directly connected, GigabitEthernet2
C 10.20.1.0/24 is directly connected, GigabitEthernet3edge2 is the mirror, with defaults via 10.20.2.1 and 10.10.2.1.
Those routes are enough for the overlay. Control connections are up to the Controller at 1.1.1.2 (both colors), the Validator at 172.16.30.30 (both colors), and the Manager at 1.1.1.1 (over mpls):
edge1#show sdwan control connections
PEER TYPE SYSTEM IP SITE PRIVATE IP LOCAL COLOR STATE
vsmart 1.1.1.2 1000 172.16.20.20 mpls up
vsmart 1.1.1.2 1000 172.16.20.20 biz-internet up
vbond 0.0.0.0 0 172.16.30.30 mpls up
vbond 0.0.0.0 0 172.16.30.30 biz-internet up
vmanage 1.1.1.1 1000 172.16.10.10 mpls upBFD is up color-to-color toward edge2 (10.10.2.12 on mpls, 10.20.2.12 on biz-internet). A VRF 10 ping to 192.168.20.10 was 5/5. The underlay defaults are doing their job.
BGP is not part of that path:
edge1#show ip bgp summary
% BGP not activeI did not start a session toward 10.10.1.1 or 10.20.1.1. Those next hops forward IP. They are not BGP speakers, so an idle neighbor would only add noise.
One more tunnel-interface detail matters if you later add PE BGP on the same ports that carry the TLOC. The running tunnel config includes allow-service all and then no allow-service bgp. The implicit ACL on a tunnel-interface can drop BGP unless you allow it. Static routing does not need that permit.
When Cisco would have you speak BGP to the PE
Cisco's BGP guide for IOS XE SD-WAN describes a site that connects directly to an L3VPN MPLS cloud. The WAN edge acts as the CE and peers with the PE. In other designs the edge is a hop away from the WAN cloud and uses BGP or OSPF toward an intermediate router so the DTLS sessions still have a route.
A stripped CE-PE session in VPN 0 looks like this (not committed):
router bgp 65000
neighbor 10.10.1.1 remote-as 65001
address-family ipv4
neighbor 10.10.1.1 activate
redistribute connectedYou would then decide what you learn: a default from the PE, more-specifics for controller addresses, or both. Keep the static defaults in place until the BGP session is established and you have checked show sdwan control connections. Removing the static default first is a good way to isolate the box from the Validator.
Some MPLS designs also put the TLOC on a loopback in VPN 0 and run the PE session toward that loopback. That is a separate, more careful underlay. I did not build it.
Cisco's overlay BGP document is easy to mix up with this one. Service-side BGP lives in a service VRF and redistributes with OMP. That is the previous BGP post. Transport-side BGP lives in VPN 0 and only has to make the TLOC addresses reachable.
Verify the underlay you actually have
| Check | Command | This lab |
|---|---|---|
| VPN 0 defaults | show ip route | Two static defaults, one per WAN next hop |
| BGP on the transport | show ip bgp summary | BGP not active |
| Control plane | show sdwan control connections | Controller, Validator, Manager up |
| Data plane | show sdwan bfd sessions | Two sessions up |
| Service traffic | ping vrf 10 192.168.20.10 | 5/5 |
What breaks
- A single static default out one color. If that next hop dies, both colors can lose controller reachability, depending on how you set routes.
- Withdrawing the static default the moment you type `router bgp`. Wait until the neighbor is Established and the control connections stay up.
- Forgetting `allow-service bgp` on a tunnel-interface that must pass CE-PE BGP. This lab currently has
no allow-service bgp. - Redistributing OMP into transport BGP, or the other way around, without a plan. VPN 0 is not a service VRF. Keep provider prefixes out of OMP unless you intend that.
- Treating this PE next hop as a place to test random BGP timers. It carries the controllers as well as the edge TLOCs.
Key takeaways
- VPN 0 needs a route to the controllers and to the far TLOCs before anything overlay-related will come up. Static defaults toward each PE are the usual way to do that.
- This lab uses two equal-cost static defaults per edge. Control connections, BFD, and a VRF 10 ping all succeed through those next hops. BGP is not active.
- Transport-side BGP is what you add when the PE expects a CE session. I did not add it, because the next hops here do not speak BGP.
- Leave the static defaults in place until a later lab has a real PE BGP speaker to measure.