In BGP, a route says "send it to 10.1.1.1". In Cisco SD-WAN, a route says "send it to 10.0.0.2, mpls, ipsec". That second answer has three parts, and new SD-WAN engineers often stare at it wondering why a next hop needs a color.
That three-part answer is a TLOC, a transport locator. Here is the short version: a TLOC is a next hop with a label telling you which pipe. This post explains the three parts, shows how TLOCs get advertised and turned into tunnels, and ends with the one keyword that decides which tunnels exist at all. I built the overlay in my home lab on Cisco Catalyst SD-WAN 20.18.2.1, with C8000v routers on IOS XE 17.18.02.
This is part of the Cisco SD-WAN series. The previous part, OMP Explained, showed that every OMP route points at a TLOC. If SD-WAN itself is new, read What is SD-WAN? first, then the components post. For the device names, the SD-WAN Controller is the box formerly called vSmart, and SD-WAN Manager (formerly vManage) doesn't appear much here. The name map has the rest.
The problem: one router, two pipes, one next hop
Your branch router has an MPLS link and an Internet link. A remote branch has the same. A normal next hop is an IP address on one interface. If that interface's address changes (DHCP on the Internet link, say), every route that used it breaks.
SD-WAN needs a next hop that:
- names the remote router, not one of its interfaces,
- says which of its transports to use, and
- says how to wrap the packet.
The idea: a TLOC is an address, plus "which door", plus "which envelope"
Think of sending a parcel to an office building. The street address gets you to the building. The door tells the courier which entrance to use, loading dock or front desk. The envelope type says how it's packed. You need all three, and the building's street address doesn't change when they repaint a door.
Cisco's routing guide defines a TLOC as a 3-tuple:
| Part | What it is | In my lab | Parcel |
|---|---|---|---|
| System IP | The router's fixed ID, like a router ID | 10.0.0.2 | Street address |
| Color | A name for the transport | mpls or biz-internet | Which door |
| Encapsulation | How the tunnel wraps the data | ipsec | Which envelope |
Cisco says it uses the system IP, not an interface IP, because "IP addresses can move or change". The guide gives two examples: DHCP, and swapped interface cards. The same guide puts it in BGP terms: the TLOC is the next hop for an OMP route and "is similar to the BGP NEXT_HOP attribute."
The parts: where a TLOC comes from and where it goes
Where it comes from: a tunnel interface
Every WAN interface with a tunnel-interface block in the transport VPN becomes one TLOC. On edge1:
edge1#show sdwan running-config sdwan
sdwan
interface GigabitEthernet2
tunnel-interface
encapsulation ipsec
color mpls restrict
...
interface GigabitEthernet3
tunnel-interface
encapsulation ipsec
color biz-internet
...edge1#show sdwan running-config system
system
system-ip 10.0.0.1
site-id 100
organization-name pinglabz-labSo edge1 has two TLOCs: (10.0.0.1, mpls, ipsec) and (10.0.0.1, biz-internet, ipsec). Cisco allows up to eight tunnel interfaces on an IOS XE edge.
What a TLOC route carries
The TLOC is the key. OMP advertises it to the Controller as a TLOC route with attributes, the most useful being the real addresses and ports the tunnel should use. Here is edge1's own mpls TLOC, as advertised to the Controller (trimmed):
edge1#show sdwan omp tlocs
---------------------------------------------------
tloc entries for 10.0.0.1
mpls
ipsec
---------------------------------------------------
ADVERTISED TO:
peer 1.1.1.2
Attributes:
encap-spi 261
encap-encrypt aes256
public-ip 10.10.1.11
public-port 12406
private-ip 10.10.1.11
private-port 12406Further up the same output, the local copy of this TLOC shows restrict 1 and bfd-status up.
The TLOC's system IP is 10.0.0.1, but the tunnel goes to 10.10.1.11 port 12406. That is the split: the TLOC is a name, and the TLOC route says where that name lives today. Public and private addresses are the same here because there's no NAT in my lab. Behind NAT they would differ. Cisco's design guide calls the private address the pre-NAT interface address and the public one the post-NAT address the Validator detects.
The Controller holds all four TLOCs in the overlay:
controller# show omp tlocs | tab
TLOC IP COLOR ENCAP FROM PEER STATUS PUBLIC IP PUBLIC PORT PRIVATE IP PRIVATE PORT
10.0.0.1 mpls ipsec 10.0.0.1 C,I,R 10.10.1.11 12406 10.10.1.11 12406
10.0.0.1 biz-internet ipsec 10.0.0.1 C,I,R 10.20.1.11 12406 10.20.1.11 12406
10.0.0.2 mpls ipsec 10.0.0.2 C,I,R 10.10.2.12 12426 10.10.2.12 12426
10.0.0.2 biz-internet ipsec 10.0.0.2 C,I,R 10.20.2.12 12426 10.20.2.12 12426(Some columns trimmed. The real table also has a TO PEER column, showing the Controller reflects each edge's TLOCs to the other edge.)
Where it goes: OMP routes point at TLOCs
A route to edge2's LAN has one path per edge2 TLOC:
edge1#show sdwan omp routes
VPN PREFIX FROM PEER PATH ID LABEL STATUS TLOC IP COLOR ENCAP
10 192.168.20.0/24 1.1.1.2 66 1004 C,I,R 10.0.0.2 mpls ipsec
1.1.1.2 68 1004 C,I,R 10.0.0.2 biz-internet ipsecTwo paths, same prefix, same system IP, different color. That is "which pipe".
The example: from TLOC to tunnel
Here is what edge1 does with what it learned. For each remote TLOC it is allowed to reach, it builds an IPsec tunnel from one of its own TLOCs and runs BFD inside it:
edge1#show sdwan bfd sessions
SYSTEM IP SITE ID STATE SOURCE TLOC COLOR REMOTE TLOC COLOR SOURCE IP DST PUBLIC IP DST PUBLIC PORT ENCAP MULTIPLIER TX INTERVAL(msec)
10.0.0.2 200 up mpls mpls 10.10.1.11 10.10.2.12 12426 ipsec 7 1000
10.0.0.2 200 up biz-internet biz-internet 10.20.1.11 10.20.2.12 12426 ipsec 7 1000The destination IP and port match edge2's TLOC route exactly. The IPsec side shows the same pairs:
edge1#show sdwan ipsec outbound-connections
SOURCE IP SOURCE PORT DEST IP DEST PORT SPI TUNNEL MTU REMOTE TLOC ADDRESS REMOTE TLOC COLOR ENCRYPTION ALGORITHM
10.10.1.11 12406 10.10.2.12 12426 262 1438 10.0.0.2 mpls AES-GCM-256
10.20.1.11 12406 10.20.2.12 12426 262 1438 10.0.0.2 biz-internet AES-GCM-256(Columns trimmed. Key hashes left out.)
Then the routing table hides all of it behind the system IP:
edge1#show ip route vrf 10
m 192.168.20.0/24 [251/0] via 10.0.0.2, 10:25:13, Sdwan-system-intfSo the chain is: prefix → TLOCs (10.0.0.2 mpls, 10.0.0.2 biz-internet) → TLOC routes (10.10.2.12:12426, 10.20.2.12:12426) → IPsec tunnels with BFD. You can see edge1 using both pipes in a traceroute from h1 to h2. One run went over mpls, a later one over biz-internet:
h1$ traceroute -n 192.168.20.10
1 192.168.10.1
2 10.20.2.12
3 192.168.20.10(The earlier run, in the components post, showed 10.10.2.12 at hop 2.)
Why only two tunnels, not four?
Two edges with two TLOCs each could make four tunnels: mpls-mpls, biz-biz, mpls-biz and biz-mpls. My lab has two. The reason is one word in the config: color mpls restrict.
Cisco's network guide says a TLOC marked restrict only builds tunnels to remote TLOCs with the same color. Both edges have restrict on mpls. So:
| edge1 TLOC | edge2 TLOC | Tunnel? | Why |
|---|---|---|---|
| mpls | mpls | Yes | Same color |
| biz-internet | biz-internet | Yes | Same color |
| mpls | biz-internet | No | edge1's mpls is restricted |
| biz-internet | mpls | No | edge2's mpls is restricted |
The local properties show the restrict flag per interface. The RESTRICT/CONTROL/STUN column reads yes/yes/no on mpls and no/yes/no on biz-internet:
edge1#show sdwan control local-properties
INTERFACE PUBLIC IPv4 PUBLIC PORT ... COLOR STATE MAX CNTRL RESTRICT/CONTROL/STUN
GigabitEthernet2 10.10.1.11 12406 ... mpls up 2 yes/yes/no
GigabitEthernet3 10.20.1.11 12406 ... biz-internet up 2 no/yes/noThis is normal design. An MPLS cloud usually can't reach the Internet, so trying to build mpls-to-Internet tunnels just creates BFD sessions that never come up.
Verify TLOCs on your own overlay
| Check | Command |
|---|---|
| My TLOCs and what I advertise | show sdwan omp tlocs (look for ADVERTISED TO) |
| Every TLOC in the overlay | show omp tlocs on the Controller |
| Which TLOCs a route uses | show sdwan omp routes vpn <n> |
| Which tunnels exist | show sdwan bfd sessions |
| The restrict flag | show sdwan control local-properties |
What breaks
- TLOC is in OMP, but no BFD session. Check reachability between the two public IPs and ports. Then check colors: if either side is
restrictand the colors differ, no tunnel is expected. - Two edges, same system IP. The system IP is part of the TLOC's identity, so two edges with the same one can't be told apart. Every device needs a unique system IP.
- Private colors behind NAT. Cisco lists mpls, metro-ethernet and private1 to private6 as private colors. Two private colors connect using private addresses, so they need no NAT between the edges. If one side is public, both use public addresses. The
carriersetting is the escape hatch when two private colors must cross NAT. - Color mismatch with a hub. If a hub has only
mpls restrictand a branch has onlybiz-internet, they will never build a tunnel. OMP still lists both TLOCs.
Key takeaways
- A TLOC is (system IP, color, encapsulation). It is the next hop for every OMP route.
- The TLOC is the name. The TLOC route carries the real public and private IP and port.
- Each pair of TLOCs that is allowed to talk gets an IPsec tunnel with BFD inside.
restrictlimits a TLOC to tunnels with the same color. In my lab it turns four possible tunnels into two.
Next: VPN 0, VPN 512 and service VPNs, which turn out to be VRFs.