VRRP at Dual-Edge SD-WAN Sites

Two WAN edges at one site solve the WAN side of redundancy. They do not automatically solve the LAN side. Hosts and switches still need a default gateway, and that gateway has to move when the active edge loses its overlay.

VRRP, the Virtual Router Redundancy Protocol, is how two service-side interfaces share one virtual IP. The hosts keep pointing at that address. The two edges decide which of them owns it. In SD-WAN the extra knob that matters is tracking: VRRP has to notice that the overlay is gone, not only that the LAN cable is still plugged in.

This post explains that design on Cisco Catalyst SD-WAN 20.18.2.1 with C8000v routers on IOS XE 17.18.02, built in my home lab. I did not add a VRRP group. The two edges are different sites with different LAN subnets, and the hosts already use each edge's own interface address as their gateway.

This is part of the Cisco SD-WAN series. Previous: TLOC extension between two WAN edges. Next: Transport-side static routes and BGP to an MPLS PE.

For SD-WAN Manager (formerly vManage) and the other old names, see the name map.

The problem: the LAN does not know about OMP

Give a floor a pair of WAN edges and a shared VLAN. Each edge has an address on that VLAN. If you pick one physical address as the default gateway, every host black-holes when that edge dies, even if the second edge is healthy.

VRRP creates a third address, the virtual IP, that floats between the two routers. The hosts use the virtual IP. The edge with the higher priority is primary. The other is backup. That is the same VRRP you already know from campus switches.

The SD-WAN catch is the failure mode VRRP cannot see by default. VRRP watches the LAN interface it runs on. If the overlay dies (all OMP sessions down) while the LAN interface stays up, the primary still answers ARP for the virtual IP, and user traffic walks into a router that can no longer forward across the fabric.

Think of two front doors that share one street number. If the occupant behind the first door loses the road out of town, someone has to take the number off that door. Tracking OMP is that hand-off.

What this lab already proves without VRRP

VPN 10 is up on both edges. Each LAN is a separate subnet. edge1 owns 192.168.10.1 on GigabitEthernet4. edge2 owns 192.168.20.1. The far prefix arrives as an OMP route:

edge1#show ip route vrf 10
C        192.168.10.0/24 is directly connected, GigabitEthernet4
m     192.168.20.0/24 [251/0] via 10.0.0.2

A ping from edge1 in VRF 10 to the host at 192.168.20.10 was 5/5, round-trip 2 to 5 ms. The local host at 192.168.10.10 replied 3/3. The overlay and the two LAN gateways work. They are not a VRRP pair.

edge1#show running-config interface GigabitEthernet4
interface GigabitEthernet4
 vrf forwarding 10
 ip address 192.168.10.1 255.255.255.0
 negotiation auto

show running-config | include vrrp matches only fhrp version vrrp v3. That line enables VRRPv3 syntax on IOS XE. It is not a group. show vrrp all printed no groups on either edge.

Cisco is explicit that VRRP belongs on service VPNs. VPN 0 and VPN 512 do not support it. The virtual IP must be different from both routers' interface addresses, and it must sit in the same subnet.

I left the gateways as they are. Moving both edges onto one LAN, changing 192.168.10.1 into a virtual IP, and giving each box a new real address would break the host default routes this series has been using.

How Cisco adds VRRP and OMP tracking

On IOS XE the service-side CLI looks like this (Cisco's documented example, not committed here):

interface GigabitEthernet0/0/1
 vrf forwarding 10
 ip address 192.168.23.2 255.255.255.0
 vrrp 1 address-family ipv4
  timers advertise 1000
  priority 200
  vrrpv2
  address 192.168.23.1 primary

The backup edge uses the same group and the same virtual IP, with a lower priority (150 in that example) and its own interface address, such as 192.168.23.3.

Two tracking options sit on top of that group:

• Track OMP. If the primary loses all of its OMP sessions, VRRP fails over to an edge that still has at least one. Failover waits for the OMP hold timer. Cisco also notes a return race: when OMP comes back, the local VRRP interface can reclaim primary before it has installed OMP routes, and overlay traffic drops until those routes land.
• Track a prefix list. VRRP watches OMP and a list of remote prefixes. If every prefix in the list disappears, failover happens without waiting for the OMP hold timer.

From Cisco IOS XE Catalyst SD-WAN Release 17.18.1a, enabling Track OMP writes track omp decrement 10 instead of track omp shutdown. These C8000v routers run 17.18.02, so that is the command I would expect if I turned tracking on here.

Cisco recommends an advertisement timer of 1000 ms or higher. A 100 ms timer on the feature template can fail when the LAN is busy.

You can build the same group from a VPN Interface Ethernet feature template (the VRRP tab) or from a configuration group's LAN interface (Add VRRP IPv4). I did not attach either. The edges are still CLI-managed (vManaged false).

Verify what you can without a group

If you do build the pair later, add show vrrp all and show vrrp detail and confirm the virtual IP, priority, and OMP track state before you change a host's default gateway.

What breaks

• Using the current interface IP as the virtual IP. Cisco requires the VIP to be different from both real addresses. The hosts here already point at .1. You would have to move the real addresses first.
• Two edges in different site-ids sharing one VLAN. That is not the dual-edge site VRRP is meant for, and it fights the OMP site model used by the BGP loop post.
• Skipping OMP tracking. A primary that lost the overlay still owns the gateway.
• A 100 ms advertise timer on a busy LAN. Cisco documents VRRP failures in that case.
• Testing this by merging the two branch LANs. Do not do that on the running two-site overlay.

Key takeaways

• VRRP gives a dual-edge site one LAN gateway. Track OMP (or a prefix list) so that gateway moves when the overlay dies, not only when the LAN interface dies.
• On 17.18.1a and later, Track OMP decrements priority by 10 instead of shutting the group down.
• This lab's edges are separate sites with separate LAN subnets and no VRRP group. The service VPN still forwards, and show vrrp all is empty.
• Keep GigabitEthernet4 as a simple gateway until you have two edges on one VLAN and a virtual IP the hosts can adopt.