SD-WAN

SD-WAN Routing Explained: Underlay, Overlay, and OMP

SD-WAN routing OMP overlay feature image, PingLabz
Table of Contents
In: SD-WAN, Fundamentals

SD-WAN routing confuses people because there are really two routing problems happening at once, and they are easy to mix up. There is the routing inside the SD-WAN fabric - how sites learn each other's prefixes across the overlay - and there is the routing of the physical transports underneath. Keep those two layers separate and SD-WAN routing becomes straightforward. This post walks through both, using the Cisco Catalyst SD-WAN (formerly Viptela) model.

For the cluster overview, see the SD-WAN complete guide.

Two layers: underlay and overlay

Every SD-WAN design has two routing layers stacked on top of each other.

The physical transports - MPLS, broadband, LTE/5G
LayerUnderlay
What it has to do
Just provide IP reachability between site edge routers
The encrypted tunnel fabric built across the transports
LayerOverlay
What it has to do
Carry the actual site-to-site enterprise routing

The important shift in mindset: the underlay does almost no routing work. It does not need to know a single enterprise prefix. Its only job is to get an SD-WAN router's transport interface reachable to the others - usually a default route from the ISP, or a simple route over MPLS. All the real routing intelligence lives in the overlay.

TLOCs: how the fabric describes a connection

Before the overlay routing makes sense, one term: a TLOC (Transport Locator) is how SD-WAN identifies a router's attachment to a transport. A TLOC is essentially the tuple of "this router, on this transport (color), with this encapsulation." A router with an MPLS link and a broadband link has two TLOCs.

TLOCs are the next-hops of the SD-WAN world. When the fabric advertises a prefix, it advertises it as reachable via a TLOC. Steering traffic onto MPLS instead of broadband is, underneath, choosing one TLOC over another.

OMP: the control plane of the fabric

The overlay's routing protocol is OMP - the Overlay Management Protocol. If you know BGP, OMP will feel familiar: it is a single protocol that distributes everything the fabric needs, and the edge routers do not peer with each other directly. Instead, every edge router peers with a central controller (the vSmart controller), and the controller reflects routing information to everyone - the same hub-and-spoke control model as a BGP route reflector.

OMP carries three things:

  • OMP routes - the enterprise prefixes at each site, advertised with their TLOC next-hop.
  • TLOC routes - the transport attachment points themselves, with their public/private addresses.
  • Service routes - advertisements for services (firewall, IPS) that traffic can be steered through.

Because OMP runs to a central controller, policy is applied centrally. The controller can shape what each site learns and which paths it prefers, without anyone touching an edge router.

Service-side routing: getting LAN prefixes into OMP

OMP handles the fabric. It does not, by itself, know about the LAN behind each site. That is the service side - the VRF/VPN facing the local network.

On the service side, an SD-WAN edge router runs ordinary routing. It learns local subnets through connected interfaces, static routes, OSPF, or BGP with the site's LAN switches. Those learned prefixes are then redistributed into OMP so the rest of the fabric can reach them. The reverse happens too: OMP routes from other sites are redistributed back into the local OSPF or BGP so LAN devices have a path out.

So the full chain for a packet's route is: LAN routing protocol redistributes into OMP, OMP carries it across the fabric to every other site, and OMP redistributes back into each remote site's LAN routing protocol. The edge router is the translation point at both ends.

Path selection: the part that makes SD-WAN "SD"

This is where SD-WAN routing stops resembling traditional routing. Traditional routing picks one best path by metric and sends everything down it. SD-WAN picks paths per application, based on measured performance.

The edge routers continuously probe each tunnel with BFD, measuring loss, latency, and jitter. A centralized application-aware routing policy then defines SLA classes - for example, "voice needs under 150 ms latency, under 2 percent loss." Traffic for an application is sent down whichever transport currently meets its SLA. If the preferred transport degrades, that application is moved to a transport that still meets the SLA, while other traffic may stay where it is.

The result: a brownout on the MPLS circuit can move voice to broadband within a couple of seconds, automatically, with no routing reconvergence in the traditional sense. The OMP routes did not change - the policy chose a different TLOC.

The configuration shape

Most SD-WAN routing is expressed as centralized policy on the controller rather than per-router CLI, but the per-site service-side routing is recognizable IOS XE:

! Service-side VRF on a Catalyst SD-WAN edge router
vrf definition 10
 rd 1:10
 address-family ipv4
!
router ospf 10 vrf 10
 redistribute omp subnets
!
! OMP redistributes the service-side routes back into the fabric
router omp
 address-family ipv4 vrf 10
  advertise ospf

Common gotchas

Sites form tunnels but cannot reach each other's LANs
Service-side prefixes are not being redistributed into OMP, or OMP routes are not redistributed back into the LAN protocol.
No OMP routes learned at all
The edge router has no control connection to the vSmart controller - an underlay reachability or certificate problem.
Traffic uses the "wrong" transport
Application-aware routing policy, or its absence - by default OMP load-shares across equal TLOCs.
Voice quality drops but routing looks fine
No SLA class defined for voice, so it is not being steered away from a degraded transport.
A whole site is unreachable after an ISP change
Underlay default route or transport-interface addressing broke - the overlay cannot form without underlay reachability.

Key takeaways

SD-WAN routing is two layers. The underlay (MPLS, broadband, LTE) only has to provide IP reachability between edge routers and carries no enterprise prefixes. The overlay carries the real routing, using OMP - a BGP-like protocol where every edge router peers with a central controller that reflects routes, advertising prefixes via TLOC next-hops. Each site's LAN prefixes are learned by ordinary OSPF, BGP, or static routing on the service side and redistributed into OMP, then redistributed back into the LAN at remote sites. The defining feature is application-aware path selection: BFD probes measure each tunnel, and centralized SLA policy steers each application onto whichever transport currently meets its requirements.

For the SD-WAN cluster, see the SD-WAN pillar.

Written by
More from Ping Labz
MPLS L3VPN vs SD-WAN migration feature image, PingLabz
MPLS

MPLS L3VPN vs SD-WAN: When to Migrate

A decision framework for migrating from MPLS L3VPN to SD-WAN: what's different, when to migrate, the 7-step transition that works, and when the right answer is to stay.
Great! You’ve successfully signed up.
Welcome back! You've successfully signed in.
You've successfully subscribed to Ping Labz.
Your link has expired.
Success! Check your email for magic link to sign-in.
Success! Your billing info has been updated.
Your billing was not updated.