Your monitoring system tells you the link is up. The user tells you the call sounds terrible. Both are correct and neither is useful, because "up" and "usable" are different questions. IP SLA is how a Cisco router answers the second one: it generates synthetic traffic, measures what happens to it, and gives you numbers you can act on.
This article builds ICMP and UDP jitter probes on IOS XE, then wrecks the path and shows the same probes reporting the damage. It extends the IP Services cluster guide.
The second half then wires a probe to a tracked object and a static route, breaks the path two hops away, and captures the failover: the return code flipping to Timeout, %TRACK-6-STATE logging the object down, the route leaving the RIB. Every capture below is verbatim from CML labs on iol-xe running IOS XE 17.18.2.
What IP SLA Actually Does
An IP SLA operation is a scheduled, synthetic test. The router sends traffic (an ICMP echo, a UDP stream, an HTTP GET, a DNS query) at a target, measures the result, stores statistics. Nothing about it is passive. Two roles exist:
ip sla N)Generates probes, measures results, holds the statistics.ip sla responder)Timestamps the probe in and out, so the source can separate network delay from the far device's processing delay.The responder is what makes jitter and one-way latency meaningful: without it you cannot tell whether 40ms of the round trip was the network or the far device answering slowly. ICMP echo needs no responder; jitter does.
The Simple One: ICMP Echo
A scheduled ping with memory. On the source router:
R1(config)# ip sla 1
R1(config-ip-sla-echo)# icmp-echo 10.0.12.2 source-interface Ethernet0/1
R1(config-ip-sla-echo)# frequency 5
R1(config-ip-sla-echo)# threshold 200
R1(config-ip-sla-echo)# timeout 500
R1(config)# ip sla schedule 1 life forever start-time nowThree timers, three jobs, and mixing them up is the classic error:
The timeout versus threshold distinction matters enormously. A probe over threshold still counts as a success and will not bring down a tracked object. Only a timeout does. If a slow path should trigger failover, set the timeout to your tolerance.
Also note ip sla schedule. Forget it and the operation sits there doing nothing, forever, with no warning. It is the most common "why is my SLA not working" cause.
Healthy baseline on a clean link:
R1#show ip sla statistics 1
IPSLA operation id: 1
Latest RTT: 3 milliseconds
Latest operation return code: OK
Number of successes: 3
Number of failures: 0The Useful One: UDP Jitter
ICMP echo tells you reachability and round-trip time. It tells you nothing about whether a voice call will sound acceptable, because voice cares far less about average latency than about jitter (variation in delay) and loss. The udp-jitter operation sends packets at a fixed interval, measures how they arrive, and needs a responder on the far end.
R2(config)# ip sla responderR1(config)# ip sla 2
R1(config-ip-sla-jitter)# udp-jitter 10.0.12.2 5000 num-packets 20 interval 20
R1(config-ip-sla-jitter)# frequency 15
R1(config-ip-sla-jitter)# tos 184
R1(config)# ip sla schedule 2 life forever start-time nowReading that: 20 packets, 20 milliseconds apart, to UDP port 5000, repeated every 15 seconds. tos 184 is DSCP EF, how real voice is marked. Marking the probe like the traffic you care about is the entire point, because the network treats the two differently. See the QoS cluster guide for why.
On a clean link, the output is boring, which is the correct result:
R1#show ip sla statistics 2
IPSLA operation id: 2
Type of operation: udp-jitter
Latest operation return code: OK
RTT Values:
Number Of RTT: 20 RTT Min/Avg/Max: 1/2/4 milliseconds
Latency one-way time:
Source to Destination Latency one way Min/Avg/Max: 0/1/2 milliseconds
Destination to Source Latency one way Min/Avg/Max: 1/1/3 milliseconds
Jitter Time:
Source to Destination Jitter Min/Avg/Max: 0/1/1 milliseconds
Destination to Source Jitter Min/Avg/Max: 0/1/2 milliseconds
Packet Loss Values:
Loss Source to Destination: 0
Loss Destination to Source: 0
Out Of Sequence: 0 Tail Drop: 0Note what the responder buys you: separate source-to-destination and destination-to-source figures. A one-way problem (congestion in one direction only, common on asymmetric WAN links) is invisible to a ping and obvious here.
Now Break the Path
Clean numbers prove the probe works, not that it is useful. So the lab applies link conditioning to the R1-R2 link: 120ms latency, 40ms jitter, 8 percent loss. Nothing else changes. Interfaces stay up, OSPF stays adjacent, and by every traditional measure the link is "fine."
The ICMP probe notices immediately:
R1#show ip sla statistics 1
IPSLA operation id: 1
Latest RTT: 238 milliseconds
Latest operation return code: Over threshold
Number of successes: 7
Number of failures: 0Over threshold, at 238ms against our 200ms threshold, and crucially Number of failures: 0. The path is terrible and the probe is succeeding. Exactly the state where a naive "is it up" monitor reports green.
The jitter probe tells the real story:
R1#show ip sla statistics 2
IPSLA operation id: 2
Latest RTT: 250 milliseconds
RTT Values:
Number Of RTT: 17 RTT Min/Avg/Max: 214/250/305 milliseconds
Latency one-way time:
Source to Destination Latency one way Min/Avg/Max: 82/119/155 milliseconds
Destination to Source Latency one way Min/Avg/Max: 91/123/155 milliseconds
Jitter Time:
Source to Destination Jitter Min/Avg/Max: 3/21/59 milliseconds
Destination to Source Jitter Min/Avg/Max: 0/17/34 milliseconds
Packet Loss Values:
Loss Source to Destination: 1
Source to Destination Loss Periods Number: 3
Loss Destination to Source: 2
Destination to Source Loss Periods Number: 1
Out Of Sequence: 6 Tail Drop: 0Compare against the baseline and the diagnosis writes itself:
Jitter went from 1ms to an average of 21ms with peaks at 59ms, against a typical voice jitter buffer of around 30ms. Packets 59ms late are discarded by the phone whether or not the network delivered them. That is how a call sounds broken while every link light is green.
Loss periods matter too. "Loss Source to Destination: 1, Loss Periods Number: 3" means three separate events rather than one burst, and scattered single-packet loss degrades voice far more gracefully because the codec conceals isolated gaps. A percentage-loss figure hides that.
The compact view:
R1#show ip sla summary
ID Type Destination Stats Return Code Last Run
*1 icmp-echo 10.0.12.2 RTT=227 Over threshold 1 seconds ago
*2 udp-jitter 10.0.12.2 RTT=250 OK 15 seconds agoA Word on MOS and ICPIF
You will notice these in the jitter output:
Voice Score Values:
Calculated Planning Impairment Factor (ICPIF): 0
Mean Opinion Score (MOS): 0Both are zero, including under the degraded conditions. That is not a bug, and it is not the network scoring perfectly. MOS and ICPIF are only calculated for a udp-jitter codec operation (udp-jitter 10.0.12.2 5000 codec g711ulaw), which makes the router emulate the packet size and rate of a real codec. A plain udp-jitter reports 0, and you should read nothing into it.
The Other Operations
ICMP echo and udp-jitter cover most needs. The rest of the list worth knowing:
icmp-echoReachability and RTT, no responder needed. The workhorse for tracking.udp-jitterJitter, one-way latency, loss. Needs a responder.udp-jitter ... codecAs above plus a real MOS score. What you want for VoIP SLAs.httpDNS, TCP connect and transfer time, measured separately.dnsResolution time. Catches the DNS server nobody is monitoring.path-echoPer-hop RTT. Finds the hop adding the delay.Measuring Is Half the Job
A probe that nobody looks at does nothing. IP SLA becomes powerful when you wire it to something that acts on the result: an object tracker that swaps a static route, an HSRP group that changes priority, an EEM applet that raises an alert. The rest of this article does that on a second lab and captures the failover as it happens. The full dual-WAN pattern (primary plus floating backup) is built end to end in the companion article on making a static route fail over when the far end dies.
Turning a Probe Into a Tracked Object
Three iol-xe routers in a line on IOS XE 17.18.2: R1 holds the probe, R2 transits, R3 sits behind R2 with 8.8.8.0/24 on a loopback.
R1(config)# ip sla 1
R1(config-ip-sla-echo)# icmp-echo 10.0.23.2 source-ip 10.0.12.1
R1(config-ip-sla-echo)# frequency 5
R1(config)# ip sla schedule 1 life forever start-time now
!
R1(config)# track 1 ip sla 1 reachability
!
R1(config)# ip route 8.8.8.0 255.255.255.0 10.0.12.2 track 1The asymmetry is the point. The next hop is 10.0.12.2 (R2, one hop away); the probe target is 10.0.23.2, two hops past it. Probe your own next hop and you have rebuilt interface tracking with extra steps. Add delay down 10 up 30 under the track too, or one lost packet becomes a routing change.
Healthy state, captured off R1:
R1#show track 1
Track 1
IP SLA 1 reachability
Reachability is Up
2 changes, last change 00:01:26
Latest operation return code: OK
Latest RTT (millisecs) 4
Tracked by:
Static IP Routing 0
R1#show ip sla statistics 1
Latest operation return code: OK
Number of successes: 19
Number of failures: 3
R1#show ip route 8.8.8.0
Routing entry for 8.8.8.0/24
Known via "static", distance 1, metric 0
* 10.0.12.2Tracked by: Static IP Routing 0 names the subsystem that subscribed to the object; an empty list means you built a check that changes nothing. The three failures are from boot, before convergence; that counter never resets, so alarm on its rate of change or on the track state.
Watching the Failover Happen
To break it, an EEM applet on R2 shut Ethernet0/1, the interface facing R3. Nothing changed on R1: its interface stayed up, its route to the next hop stayed valid, its ARP entry stayed put. Locally nothing failed, which is the scenario a plain static route loses to.
*Jul 20 21:03:40.297: %TRACK-6-STATE: 1 ip sla 1 reachability Down -> Up
! ... R2 shuts Ethernet0/1, breaking the path to the probe target ...
*Jul 20 21:06:10.306: %TRACK-6-STATE: 1 ip sla 1 reachability Up -> DownThe first line is boot convergence. The second is the trigger. %TRACK-6-STATE is severity 6 informational, on by default and easy to scroll past. Filter for it and alert on it: it names both the object and the direction.
R1#show track 1
Reachability is Down
3 changes, last change 00:00:56
Latest operation return code: Timeout
Tracked by:
Static IP Routing 0
R1#show ip sla statistics 1
Latest RTT: NoConnection/Busy/Timeout
Latest operation return code: Timeout
Number of successes: 30
Number of failures: 16
R1#show ip route 8.8.8.0
% Network not in tableNoConnection/Busy/Timeout is one field listing three possible non-numeric outcomes, so read the return code below it instead. The counters moving 19/3 to 30/16 prove the probe never stopped polling, which matters when you debug a failover that did not happen (a probe that silently died generates no failures and never brings a track down). And % Network not in table is the mechanism proven: a device two hops away lost an interface, and a static route on a router that saw no local failure pulled itself from the RIB.
What Takes Over When the Route Is Withdrawn
In this lab, nothing did, deliberately, so the withdrawal was unambiguous. In production a second route waits, and the reason it waits is how a router chooses between two sources for the same prefix. Tracking does not perform the failover. It performs the withdrawal. Administrative distance performs the failover.
ip route 8.8.8.0 255.255.255.0 10.0.12.2 track 1
ip route 8.8.8.0 255.255.255.0 10.0.14.2 200The healthy capture read Known via "static", distance 1. That second line is a floating static at AD 200: invisible while the AD 1 route exists, best remaining candidate the instant the track pulls it. Pick the distances carelessly (200 for the backup when an EIGRP path at 90 also carries the prefix) and the failover lands somewhere you did not intend.
Tracking Things That Are Not Static Routes
The object is generic, and show track lists every subsystem consuming it. The common second consumer is a first hop redundancy group: standby 1 track 1 decrement 30 drops an HSRP router's priority when the probe fails. Better than tracking an interface, but a flapping probe now produces a flapping HSRP group, one of the classic causes of an HSRP pair that keeps swapping Active, so set the track delay first. The same trade-off appears on a firewall pair running stateful failover between two ASA units: monitor something close for late but rare failovers, or something distant for early ones with occasional false positives.
Common Mistakes and Gotchas
- Nothing is consuming the track. An empty
Tracked bylist is a check wired to nothing: state changes, no traffic moves. - Probing the next hop instead of past it. Same IP as the static's next hop means you reinvented interface tracking.
- Reading the cumulative failure counter as health. It never resets, and boot convergence seeds it.
- No
delayon the track. Every lost packet on a marginal circuit becomes a routing change.
FAQ
Why is my SLA showing no statistics at all?
You almost certainly forgot ip sla schedule N life forever start-time now. An unscheduled operation never runs and reports nothing.
Do I need a responder for ICMP echo?
No. Any device that answers a ping works, including a Linux host or a firewall. The responder is only for jitter and one-way measurements.
What is the difference between timeout and threshold?
Timeout produces a FAILURE and can bring down a tracked object. Threshold produces a warning while still counting as a SUCCESS. Only timeout drives failover.
How much traffic does a probe generate?
Very little, but not zero, and a high-frequency udp-jitter probe across hundreds of routers is real traffic. Size the frequency to the decision: 5 seconds for a failover trigger, 60 seconds for a trend graph.
Why is MOS always 0?
Because you are running a plain udp-jitter operation. MOS and ICPIF are only calculated for codec-based jitter operations.
My track is Down but the route is still in the table
Check show track for a Tracked by entry. If Static IP Routing is not listed, the route was configured without the track keyword and is not subscribed to the object.
Should I track reachability or state?
Use reachability to fail over on a dead path, which is the common case. Use state if a degraded-but-alive path is also unacceptable, and accept that it treats Over threshold as a failure.
Key Takeaways
- IP SLA answers "is this path usable," not "is this link up."
ip sla scheduleis mandatory. Without it the operation exists and never runs.- timeout causes a failure and drives tracking. threshold only flags a slow-but-successful probe.
- The responder enables jitter, one-way latency and loss-direction figures. ICMP echo does not need one; udp-jitter does.
- Mark the probe like the traffic you care about (
tos 184for voice), or you measure a path the real traffic will not take. - A degraded path reports
Over thresholdwithNumber of failures: 0. Green dashboards and unusable links coexist happily. - MOS and ICPIF only populate on codec-based jitter operations.
track N ip sla N reachabilityturns a return code into an object state, andTracked bynames whatever consumes it.- The captured failover needed no local failure: R2 lost an interface, R1's SLA returned
Timeout,%TRACK-6-STATElogged Up to Down, the route became% Network not in table. - Tracking withdraws a route; administrative distance decides what replaces it. Design both, and point the probe past the next hop.
Next: build the dual-WAN version with a primary and a floating backup static, or work through the rest of the router services toolkit on PingLabz.
