VXLAN with BGP EVPN: The Fabric Control Plane

VXLAN gives you the data plane: a way to wrap Ethernet in UDP and route it across a fabric. But an encapsulation is useless until the ingress switch knows which remote switch a destination lives behind. The original answer was to flood and learn, exactly like a dumb switch, which does not scale to a real data centre. The modern answer is to give VXLAN a proper control plane, and that control plane is MP-BGP with the EVPN address family.

This article explains BGP EVPN: what it advertises, the route types that carry it, and why it replaced flood-and-learn everywhere. It builds on the VXLAN deep dive and the BGP cluster, and extends the Network Virtualization cluster guide.

A Note on the Captures

Consistent with this site's rule that captured output is always real, the VXLAN data-plane bytes in this cluster come from a genuine crafted packet (see the VXLAN deep dive, which shows the real 4789/UDP encapsulation and VNI 10010 on the wire). The EVPN control-plane configuration and show-command structure below are presented as reference syntax drawn from Cisco's documented NX-OS behaviour, not as lab captures, because a virtual Nexus fabric could not be brought to a usable login state in this build. Where this article shows a show command, treat it as the command to run and the fields to read, not as a claim of captured output. The concepts, route types, and configuration are accurate and standard.

Why Flood-and-Learn Had to Go

In flood-and-learn VXLAN, a VTEP learns remote MACs the way a switch does: it floods unknown-unicast, broadcast, and multicast (BUM) traffic to every other VTEP, watches the replies, and builds its mapping table from the data plane. This works, and it is simple, but it has the same problems that made large flat Layer 2 networks a bad idea:

Flooding does not scale. Every unknown destination triggers fabric-wide flooding. In a big fabric that is a lot of wasted bandwidth and a lot of BUM replication.
It needs multicast in the underlay (or ingress replication), which is operationally heavy. See the multicast cluster for why running PIM everywhere is not free.
No proactive knowledge. A VTEP knows nothing about a host until traffic forces it to learn, so the first packet of every new conversation floods.

BGP EVPN fixes all three by turning reachability into a routing problem: VTEPs advertise the MACs and IPs they have, so every other VTEP already knows where everything is before the first packet.

EVPN Is an Address Family

EVPN is not a new protocol. It is an address family of MP-BGP (the same MP-BGP that carries MPLS L3VPN VPNv4 routes), called l2vpn evpn. If you understand how MP-BGP carries VPNv4 prefixes with route distinguishers and route targets, you already understand 80% of EVPN, because it uses the identical machinery for a different payload.

What it carries is a set of route types, each advertising a different kind of reachability:

Type 2: MAC/IP

"Host with this MAC (and this IP) lives behind me." The workhorse. Advertises endpoint reachability and populates remote ARP/MAC tables without flooding.

Type 3: IMET

"I am a VTEP participating in this VNI." Builds the flood list for the BUM traffic that still must be replicated (like broadcasts), so ingress replication needs no multicast.

Type 5: IP Prefix

"This IP prefix is reachable via me." Carries routed prefixes (L3VNI) and external/summary routes into the fabric, for inter-subnet and outside connectivity.

Type 1 & 4

Ethernet Auto-Discovery and Ethernet Segment routes, used for multihoming (ESI/VPC) and fast convergence. Important at scale, secondary when learning the basics.

The two you must know cold are Type 2 (a host is here) and Type 5 (a subnet is here). Type 2 is how the fabric learns endpoints without flooding; Type 5 is how it does routing between subnets and reaches the outside world.

The Type 2 Trick: No More ARP Flooding

Here is the single most elegant thing EVPN does. A Type 2 route advertises both the MAC and the IP of a host. So when host A wants to talk to host B and sends an ARP request for B's IP, A's local leaf already has a Type 2 route telling it B's MAC. The leaf answers the ARP locally (ARP suppression) instead of flooding it across the fabric.

Multiply that across thousands of hosts and you have eliminated fabric-wide ARP flooding entirely. The endpoints think they are on a normal broadcast segment; the fabric never actually broadcasts their ARP. That is the scalability win, and it is impossible with flood-and-learn.

The Configuration Shape

Reference configuration for a leaf VTEP (NX-OS style). Read it for the structure; the pieces map directly onto the concepts above:

! features
feature ospf
feature bgp
feature nv overlay
feature vn-segment-vlan-based
nv overlay evpn
!
! the L2VNI to VLAN mapping
vlan 10
  vn-segment 10010
!
! the NVE interface - the VTEP itself
interface nve1
  source-interface loopback0
  host-reachability protocol bgp        ! use EVPN, not flood-and-learn
  member vni 10010
    ingress-replication protocol bgp     ! BUM via EVPN Type 3, no multicast
!
! MP-BGP with the EVPN address family
router bgp 65000
  neighbor 10.0.0.100 remote-as 65000    ! the spine / route reflector
    address-family l2vpn evpn
      send-community extended
!
! tie the VLAN into EVPN
evpn
  vni 10010 l2
    rd auto
    route-target import auto
    route-target export auto

The lines that matter conceptually: host-reachability protocol bgp is what turns off flood-and-learn and turns on EVPN. ingress-replication protocol bgp is what lets BUM traffic be replicated using the Type 3 flood list instead of underlay multicast. And rd auto / route-target ... auto is the same RD/RT construct from MPLS L3VPN, here keeping tenants separate in the EVPN table.

The Commands You Would Run

On a live fabric, these are the verification commands and what each one tells you:

show nve peersWhich remote VTEPs this leaf has discovered (learned from Type 3 routes). The overlay adjacency list.

show bgp l2vpn evpnThe EVPN table itself: every Type 2, 3, and 5 route, with its RD and route targets. This is where you confirm a host was advertised.

show l2route evpn mac allThe MACs learned via EVPN and which VTEP each is behind. The control-plane MAC table.

show nve vniWhich VNIs are up on this VTEP and their state. Your first check when a segment is not passing traffic.

When a host in VNI 10010 on one leaf cannot reach a host on another, the troubleshooting path is: is the VNI up (show nve vni), is the remote VTEP a peer (show nve peers), and did the host's Type 2 route arrive (show bgp l2vpn evpn). Missing at the last step usually means a route-target or RD mismatch, the exact same failure mode as MPLS L3VPN.

Why the Spine-Leaf Topology

EVPN fabrics are built spine-leaf for a reason that ties back to BGP. Leaves are the VTEPs (where hosts connect); spines are the transit and the BGP route reflectors. Every leaf peers EVPN with the spines, and the spines reflect routes between leaves, so you get a full mesh of reachability without a full mesh of BGP sessions, the same route-reflector scaling trick used in any large iBGP network. The underlay (OSPF or eBGP) provides loopback-to-loopback reachability between VTEPs; the overlay (EVPN) rides on top. Two BGP roles, one physical fabric.

FAQ

Is EVPN a replacement for VXLAN?

No. VXLAN is the data-plane encapsulation; EVPN is the control plane that tells VTEPs where to send. They work together. "VXLAN-EVPN" is the full stack.

What are the two route types I must know?

Type 2 (MAC/IP, advertises a host and enables ARP suppression) and Type 5 (IP prefix, carries routed subnets and external routes). Type 3 (IMET) builds the BUM flood list.

Does EVPN need multicast?

No. That is a key advantage over flood-and-learn. BUM traffic is handled by ingress replication driven by Type 3 routes, so the underlay only needs unicast routing.

Very closely. EVPN is an MP-BGP address family using the same RD/RT machinery as VPNv4. If you know L3VPN, EVPN's control plane will feel familiar; the payload is MACs and host IPs instead of VPN prefixes.

What is symmetric IRB?

The standard routing model where inter-subnet traffic is routed at the ingress leaf into an L3VNI and carried across the fabric in that VNI, with every leaf sharing an anycast gateway. It is what makes a host's gateway always local, enabling seamless mobility.

Key Takeaways

VXLAN is the data plane; BGP EVPN is the control plane that replaced flood-and-learn. Together they are the modern fabric.
Flood-and-learn does not scale (fabric-wide flooding, needs multicast). EVPN advertises reachability proactively, so VTEPs know where hosts are before the first packet.
EVPN is an MP-BGP address family (l2vpn evpn) using the same RD/RT machinery as MPLS L3VPN.
Know Type 2 (MAC/IP, enables ARP suppression, no ARP flooding) and Type 5 (IP prefix, routing and external). Type 3 builds the BUM flood list.
Key config: host-reachability protocol bgp (turns on EVPN) and ingress-replication protocol bgp (BUM without multicast).
Verify with show nve peers, show bgp l2vpn evpn, show l2route evpn mac all, show nve vni. A missing Type 2 route is usually an RD/RT mismatch, just like L3VPN.
Spine-leaf exists because spines are the BGP route reflectors, scaling the overlay the same way route reflectors scale any iBGP network.

Next: Cisco SD-Access architecture (which combines LISP and VXLAN), or the Network Virtualization cluster guide.