An edge router boots in a branch. It has an IP address, a certificate and the address of one Validator. A few seconds later it trusts the Controller, takes config from SD-WAN Manager and builds tunnels to other sites. How did it get from one address to all of that, safely?
Through control connections. This post walks through the order they come up, what DTLS is doing on the wire, and how to read the history when one fails. I cleared the control connections on a live edge and captured the rebuild with tcpdump. I built the overlay in my home lab on Cisco Catalyst SD-WAN 20.18.2.1, with C8000v routers on IOS XE 17.18.02.
This is part of the Cisco SD-WAN series. The previous part, VPN 0, VPN 512 and Service VPNs, covered the transport VPN these connections live in. For the old names of SD-WAN Manager (formerly vManage), the Controller and the Validator, see the name map.
The problem: trust over an untrusted network
The edge talks to its controllers across the Internet or a carrier network. Anyone could sit in the path. So every control conversation has to answer two questions before anything else:
- Is the other side really my controller?
- Can anyone else read or change what we say?
A plain TCP session answers neither.
The idea: show your badge at the door
Think of a secure office. You show your badge at reception. The guard checks it against the company list and shows you their own badge. Only then do you get a visitor pass and directions to the right floors.
In SD-WAN the badge is a certificate. Reception is the Validator. The floors are SD-WAN Manager and the Controllers. And the conversation happens inside DTLS, which is TLS running over UDP.
The parts
The order
According to Cisco's docs, an edge brings up its controllers in this order:
- Validator. The only address the edge knows. They check each other's certificates. The Validator then tells the edge where SD-WAN Manager and the Controllers are, and which public address it saw the edge come from.
- SD-WAN Manager. The edge gets its config.
- Controllers. The edge starts OMP and learns routes and TLOCs.
Once the edge has a Controller, the Validator's job is mostly done.
DTLS or TLS
DTLS is the default. It's TLS for UDP, with its own fix for lost packets. TLS over TCP is an option on the Controller and Manager. Validator connections always use DTLS, according to Cisco's docs.
My edge shows the default:
edge1#show sdwan control local-properties
protocol dtls
tls-port 0
retry-interval 0:00:00:18
no-activity-exp-interval 0:00:00:20Ports
Controllers listen on UDP 12346. Cisco's docs say each extra CPU core on a controller listens on a port 100 higher: 12446, 12546 and so on. My edges connected to SD-WAN Manager on 12746, and edge2 landed on 12946. That's the same Manager, different cores.
The Validator shows it plainly. Every controller core keeps its own DTLS session to the Validator (trimmed):
validator# show orchestrator connections | tab
PEER TYPE PEER PUBLIC IP PORT STATE
vsmart 172.16.20.20 12346 up
vsmart 172.16.20.20 12446 up
vmanage 172.16.10.10 12346 up
vmanage 172.16.10.10 12446 up
... (12546 to 12946)
vmanage 172.16.10.10 13046 upTwo ports for my Controller, eight for my Manager.
Edges start at 12346 too and hop to a new port (12366, 12386 and so on) when connections fail. My edge1 sits on 12406 and edge2 on 12426. They've hopped a few times since boot.
The lab: tear down and watch the rebuild
Here is edge1 before the test, with one connection per controller per color (columns trimmed):
edge1#show sdwan control connections
PEER TYPE PROT SYSTEM IP PRIVATE IP PORT LOCAL COLOR STATE
vsmart dtls 1.1.1.2 172.16.20.20 12346 mpls up
vsmart dtls 1.1.1.2 172.16.20.20 12346 biz-internet up
vbond dtls 0.0.0.0 172.16.30.30 12346 mpls up
vbond dtls 0.0.0.0 172.16.30.30 12346 biz-internet up
vmanage dtls 1.1.1.1 172.16.10.10 12746 mpls upvsmart is the Controller, vbond the Validator and vmanage SD-WAN Manager. The CLI still uses the old names.
I started tcpdump on both WAN links, filtered on UDP 12346 and 12746. Then I ran clear sdwan control connections on edge1.
What the wire showed
tshark decoded the biz-internet link like this (timestamps in seconds from capture start):
4.766 10.20.1.11 -> 172.16.30.30 Encrypted Alert
4.770 10.20.1.11 -> 172.16.20.20 Encrypted Alert
4.825 10.20.1.11 -> 172.16.30.30 Client Hello
4.875 172.16.30.30 -> 10.20.1.11 Hello Verify Request
4.924 10.20.1.11 -> 172.16.30.30 Client Hello
4.926 172.16.30.30 -> 10.20.1.11 Server Hello, Certificate
4.945 172.16.30.30 -> 10.20.1.11 Certificate, Server Key Exchange,
Certificate Request, Server Hello Done
5.064 10.20.1.11 -> 172.16.30.30 Certificate, Client Key Exchange,
Certificate Verify, Change Cipher Spec
5.070 172.16.30.30 -> 10.20.1.11 Change Cipher Spec, Finished
5.168 10.20.1.11 -> 172.16.20.20 Client Hello
... (same pattern to the Controller, done at 5.405)Read it top to bottom:
- Encrypted Alert. The clear command ended the old sessions with an encrypted DTLS alert to each controller. It's encrypted, so the capture can't show which alert, but a close notice is the likely one.
- Client Hello to the Validator first. 172.16.30.30 is the Validator. The edge goes there before anyone else.
- Hello Verify Request. This is DTLS's own step. The server sends back a cookie and the client must repeat its hello with it. It stops someone flooding a server with spoofed UDP hellos.
- Certificate both ways. The Validator sends its certificate and a Certificate Request. The edge answers with its own. Both sides prove who they are. That's mutual authentication.
- Change Cipher Spec. From here, everything is encrypted.
- Then the Controller. Only after the Validator handshake finished did the edge start on 172.16.20.20.
On the mpls link the same thing happened, plus a third handshake to SD-WAN Manager on 12746. It ran alongside the Controller handshake. The whole rebuild took about 0.6 seconds per link.
The detailed decode showed the chosen cipher and the certificate:
Cipher Suite: TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384 (0xc030)
Named Curve: secp521r1
issuer: commonName=PingLabz Lab Root CA, organizationName=pinglabz-labThe organization name in the certificate matters. Every device checks that the peer belongs to the same organization, pinglabz-lab here.
After the rebuild
The history (columns trimmed) logged why each one went down:
edge1#show sdwan control connection-history
PEER TYPE LOCAL COLOR STATE LOCAL ERR REMOTE ERR DOWNTIME
vsmart biz-internet tear_down DISTLOC NOERR 2026-11-05T16:42:04
vbond biz-internet tear_down DISTLOC NOERR 2026-11-05T16:42:04
vsmart mpls tear_down DISTLOC NOERR 2026-11-05T16:42:04
vmanage mpls tear_down DISTLOC NOERR 2026-11-05T16:42:04
vbond mpls tear_down DISTLOC NOERR 2026-11-05T16:42:04DISTLOC means the local side tore the connection down, which is what I did. All five came back up within a minute.
One side effect: show sdwan bfd summary showed the flap counter going from 2 to 4. The data-plane tunnels flapped too, since the edge lost its Controller for a moment. Don't run this command on a production edge casually.
The history tells the story
The same history held the first boot of this lab, hours earlier. It's a good example of a real failure (columns trimmed, last column is the count):
vbond mpls connect DCONFAIL NOERR 19
vbond mpls challenge_resp RXTRDWN SERNTPRES 8
vbond mpls up NOERR NOERR 1- DCONFAIL 19 times: DTLS connection failure. The edge couldn't complete a handshake with the Validator, typically because it isn't reachable yet.
- SERNTPRES 8 times, from the remote side: the device serial number isn't present on the Validator. The Validator didn't have the edge on its allowed list yet.
- up: once the device list synced, the connection came up.
The legend for every code prints at the top of show sdwan control connection-history.
Verify control connections on your own edges
| Check | Command |
|---|---|
| Current connections | show sdwan control connections |
| Protocol, ports, timers, per-color limits | show sdwan control local-properties |
| Failures with error codes | show sdwan control connection-history |
| The Validator's side | show orchestrator connections on the Validator |
| The handshake itself | tcpdump on UDP 12346, decode as DTLS in Wireshark |
What breaks
- DCONFAIL. No DTLS session. Check reachability to the Validator and that UDP 12346 is open both ways.
- SERNTPRES or BIDNTPR. The Validator doesn't know the device or its certificate. Sync the device list from SD-WAN Manager.
- CRTVERFL. Certificate check failed, often because the root CA isn't installed or the clock is wrong. Certificates have dates, so check NTP.
- Organization mismatch. The org name in config must match the certificate exactly, including case.
- Firewall allows 12346 only. Edges hop ports and Manager cores listen on higher ones. Cisco's firewall guide lists the full range.
Key takeaways
- Control connections are DTLS by default: TLS over UDP, with mutual certificate checks.
- The edge talks to the Validator first. My capture showed the Controller and Manager handshakes starting only after the Validator one finished.
- DTLS adds a Hello Verify Request cookie step that TLS doesn't have.
- My lab negotiated TLS_ECDHE_RSA_WITH_AES_256_GCM_SHA384, with certificates from my own root CA.
show sdwan control connection-historyand its error codes tell you which step failed.