Path MTU problems have a signature that wastes more engineer hours than almost anything else: small things work and big things do not. SSH connects and then hangs the moment you run a command that prints a screenful. Web pages load their HTML and stall on an image. A VPN comes up and then nothing passes. Ping is clean, traceroute is clean, and every monitoring dashboard is green, because everything anyone tested was small.
tracepath is the tool that finds it in one command. It walks the path like traceroute does, but it also probes for the largest packet that survives and tells you where the path narrows. This article shows a healthy 1500 byte path, then lowers the MTU on one router to router link and watches tracepath discover it, then breaks path MTU discovery entirely to show what a black hole looks like from the client. All output is real, captured on a Debian 13 host through a Cisco Modeling Labs topology, and this article is part of the Linux networking commands cluster.
A clean path
With a uniform 1500 byte MTU end to end:
j@llmbits:~$ tracepath -n 10.77.3.10
1?: [LOCALHOST] pmtu 1500
1: 10.77.0.1 3.253ms
1: 10.77.0.1 2.409ms
2: 10.77.12.2 3.667ms
3: 10.77.23.3 4.725ms
4: 10.77.3.10 5.213ms reached
Resume: pmtu 1500 hops 4 back 4 Four things in that output are worth naming.
The 1?: [LOCALHOST] pmtu 1500 line is the local interface MTU, the starting assumption. Everything after it is measured.
Hop 1 appears twice. That is not a bug. tracepath sends an initial probe to establish the local MTU and prints the first hop again once it starts the real walk.
reached means the destination answered, which tracepath detects the same way UDP traceroute does, via an ICMP port unreachable.
The Resume line is the summary and the only line most people need: path MTU 1500, four hops out, and back 4, which is the TTL arithmetic on the returning packet. When hops and back disagree, the forward and return paths have different lengths, which is a quiet way of spotting asymmetric routing.
Unlike traceroute's ICMP and TCP modes, tracepath needs no privileges at all. There is no sudo in front of any tracepath command in this article. It uses an ordinary UDP socket and asks the kernel to do path MTU discovery on it, which is why it is often the only path tool available to an unprivileged service account.
How path MTU discovery works
Linux sets the Don't Fragment bit on outbound IPv4 packets by default. When such a packet is too large for the next link, the router cannot fragment it and cannot forward it, so it drops the packet and returns ICMP type 3, code 4, destination unreachable with the code "fragmentation needed and DF set." Since RFC 1191 that message also carries the MTU of the next hop, which turns the error into a useful instruction.
The sending host caches that number against the destination and starts sending smaller packets. IPv6 has no fragmentation by routers at all, so this mechanism is not an optimization there, it is the only thing that works.
The whole scheme depends on one ICMP message getting back to the sender. That is its strength (one round trip and you know) and its weakness (any firewall that blocks ICMP breaks it silently).
Narrowing one link
Now ip mtu 1400 is configured on both ends of the R2 to R3 link. On IOS this changes the IP MTU without touching the interface MTU, which is worth seeing because the two are separate numbers:
R2#show ip interface Ethernet0/1 | include MTU|Internet address
Internet address is 10.77.23.2/24
MTU is 1400 bytes
R2#show interfaces Ethernet0/1 | include MTU
MTU 1500 bytes, BW 10000 Kbit/sec, DLY 1000 usec,The interface still carries 1500 byte frames. The IP layer will only forward datagrams up to 1400. show interfaces and show ip interface report different numbers for the same port, and a great deal of MTU troubleshooting goes wrong because somebody read the first one.
From the Linux side, after flushing the route cache:
j@llmbits:~$ tracepath -n 10.77.3.10
1?: [LOCALHOST] pmtu 1500
1: 10.77.0.1 3.479ms
1: 10.77.0.1 2.515ms
2: 10.77.12.2 3.689ms
3: 10.77.12.2 3.354ms pmtu 1400
3: 10.77.23.3 4.361ms
4: 10.77.3.10 5.072ms reached
Resume: pmtu 1400 hops 4 back 4 Read line by line. At TTL 3, the router at 10.77.12.2 reports back pmtu 1400. That is R2 saying "the packet you want me to send out of Ethernet0/1 is too big, and the limit is 1400." tracepath prints the hop that rejected the packet, not the hop that follows it, so the constrained link is the one leaving the address shown. Then tracepath drops its probe size and immediately gets an answer from 10.77.23.3 at the same TTL, and continues.
The Resume line now reads 1400. In one command, on an unprivileged shell, you have both the narrow point and the number.
Where Linux stores the answer
The discovered MTU is not a property of the interface, it is cached per destination in the routing cache:
j@llmbits:~$ ip route get 10.77.3.10
10.77.3.10 via 10.77.0.1 dev ens224 src 10.77.0.100 uid 1000
cache expires 598sec mtu 1400 Compare that to the same command before the discovery, where the cache line was bare:
j@llmbits:~$ ip route get 10.77.3.10
10.77.3.10 via 10.77.0.1 dev ens224 src 10.77.0.100 uid 1000
cache ip route get is the fastest MTU check on a Linux box and almost nobody uses it for that. If the cache line shows an mtu value below your interface MTU, path MTU discovery has already fired for that destination and you have your answer without sending anything. For the rest of what ip route get can tell you, see managing the Linux routing table.
The relevant tunables:
j@llmbits:~$ sysctl net.ipv4.ip_no_pmtu_disc net.ipv4.route.mtu_expires net.ipv4.route.min_pmtu net.ipv4.tcp_mtu_probing
net.ipv4.ip_no_pmtu_disc = 0
net.ipv4.route.mtu_expires = 600
net.ipv4.route.min_pmtu = 552
net.ipv4.tcp_mtu_probing = 0mtu_expires is 600 seconds, which is the ten minute countdown in that cache line. After it lapses the host retries at full size, which is how a path that has been repaired recovers on its own. It is also why a cached small MTU can outlive the fix by ten minutes and make you think you did not fix anything. sudo ip route flush cache clears it immediately.
Confirming with ping
ping -M do sets the Don't Fragment bit and refuses to fragment locally, which turns ping into an MTU measuring instrument. The -s value is payload, so add 28 bytes for the IP and ICMP headers to get the on the wire size.
On the 1400 byte path, a 1500 byte packet fails and a 1400 byte packet succeeds:
j@llmbits:~$ ping -M do -s 1472 -c 2 10.77.3.10
PING 10.77.3.10 (10.77.3.10) 1472(1500) bytes of data.
ping: sendmsg: Message too long
ping: sendmsg: Message too long
--- 10.77.3.10 ping statistics ---
2 packets transmitted, 0 received, +2 errors, 100% packet loss, time 1020ms
j@llmbits:~$ ping -M do -s 1372 -c 2 10.77.3.10
PING 10.77.3.10 (10.77.3.10) 1372(1400) bytes of data.
1380 bytes from 10.77.3.10: icmp_seq=1 ttl=61 time=8.07 ms
1380 bytes from 10.77.3.10: icmp_seq=2 ttl=61 time=6.71 ms
--- 10.77.3.10 ping statistics ---
2 packets transmitted, 2 received, 0% packet loss, time 1001ms
rtt min/avg/max/mdev = 6.714/7.390/8.066/0.676 msNote precisely what failed. sendmsg: Message too long is a local error. The packet never left the host, because the kernel already knew from the cached PMTU that 1500 would not fit and refused to send it. That local refusal is the sign of a working path MTU discovery. Hold onto it, because the next section is what happens when it is not working. Full coverage of DF bit testing lives in ping, MTU and the DF bit.
The black hole
Now the failure mode that actually costs people days. A firewall somewhere drops inbound ICMP unreachables, so the fragmentation needed message never comes home. Simulated here by dropping exactly that one message type at the client:
j@llmbits:~$ sudo iptables -I INPUT -p icmp --icmp-type fragmentation-needed -j DROP
j@llmbits:~$ sudo iptables -L INPUT -n --line-numbers | head -5
Chain INPUT (policy ACCEPT)
num target prot opt source destination
1 DROP icmp -- 0.0.0.0/0 0.0.0.0/0 icmptype 3 code 4Flush the route cache so the host forgets what it learned, and try again:
j@llmbits:~$ sudo ip route flush cache
j@llmbits:~$ timeout 45 tracepath -n 10.77.3.10
j@llmbits:~$ Nothing. Not a partial path, not an error, no output at all in 45 seconds before it was killed. tracepath sends a large probe, the router drops it, the ICMP that would explain why is discarded at the client, and tracepath sits waiting for an answer that has already been thrown away.
j@llmbits:~$ ip route get 10.77.3.10
10.77.3.10 via 10.77.0.1 dev ens224 src 10.77.0.100 uid 1000
cache The cache is bare. Nothing was learned. And the ping that produced a clear local error a moment ago now produces the least helpful output in networking:
j@llmbits:~$ ping -M do -s 1472 -c 3 -W 2 10.77.3.10
PING 10.77.3.10 (10.77.3.10) 1472(1500) bytes of data.
--- 10.77.3.10 ping statistics ---
3 packets transmitted, 0 received, 100% packet loss, time 2027msOne hundred percent loss and no error message, because this time the packets really did leave the host. Compare the two failures side by side, because telling them apart is the whole skill:
sendmsg: Message too longmtu 1400cache, nothing learnedRemove the rule and everything comes back at once:
j@llmbits:~$ sudo iptables -D INPUT -p icmp --icmp-type fragmentation-needed -j DROP
j@llmbits:~$ sudo ip route flush cache
j@llmbits:~$ tracepath -n 10.77.3.10
1?: [LOCALHOST] pmtu 1500
1: 10.77.0.1 3.501ms
1: 10.77.0.1 2.505ms
2: 10.77.12.2 3.925ms
3: 10.77.12.2 3.549ms pmtu 1400
3: 10.77.23.3 4.941ms
4: 10.77.3.10 5.648ms reached
Resume: pmtu 1400 hops 4 back 4 If you take one operational rule from this article, take this one: never blanket deny ICMP inbound. Permit type 3 code 4 at minimum. Every firewall vendor's default templates permit it, and it gets removed by hand, by people tightening rules who do not know what that one line does.
The workaround when you cannot fix the firewall
Sometimes the device eating the ICMP belongs to somebody who will not change it. net.ipv4.tcp_mtu_probing is the Linux answer, and it was 0 (off) on this host. Set to 1, the kernel leaves normal PMTUD alone until a connection stalls in a way that looks like a black hole, then starts probing downward on its own. Set to 2 it always uses probing.
sudo sysctl -w net.ipv4.tcp_mtu_probing=1Two caveats. It only helps TCP, so UDP based tunnels and QUIC are untouched. And it is a workaround, not a fix: connections still stall for a few seconds before the probing kicks in. It belongs on hosts that live behind somebody else's broken firewall, not on your own network where you can fix the actual problem.
The other common fix is clamping TCP MSS on the router so the endpoints never negotiate a segment size that will not fit. That is standard practice on tunnel interfaces, and it is why ip tcp adjust-mss appears in almost every GRE and IPsec template.
tracepath, traceroute and ping
--mtuThe numbers worth knowing by heart
FAQ
tracepath prints nothing at all. What now?
That is the black hole signature above. Something on the path is dropping oversized packets and something is dropping the ICMP that would explain it. Work inward: run tracepath to each hop you know about until one of them answers, and the first hop that does not is on the far side of the problem.
Does tracepath work for IPv6?
Yes, as tracepath -6 or as the separate tracepath6 binary on older distributions. It matters more for IPv6 than IPv4, because IPv6 routers never fragment and the packet too big message is the only mechanism available. An IPv6 path with ICMPv6 filtered does not degrade, it fails.
Why does hop 1 always appear twice?
The first probe establishes the local MTU and is reported against the local interface, then the walk proper begins at TTL 1 again. It is cosmetic and it appears in every tracepath run.
What is the difference between MTU and MSS?
MTU is the largest IP datagram a link will carry, headers included. MSS is the largest TCP payload, negotiated between the endpoints in the SYN, and it is normally MTU minus 40 bytes for the IPv4 and TCP headers. On the 1400 byte path in this article the endpoints settled on an MSS of 1348, and you can see that number in ss -i output on any live connection.
Should I just lower the MTU on all my hosts?
It is a blunt instrument that works and costs throughput on every path that did not need it. Prefer fixing the ICMP filtering, then MSS clamping on the tunnel, then host MTU as a last resort. Lowering the MTU on a laptop is a fine emergency fix for one user; lowering it fleetwide to paper over one broken firewall is not.
Key takeaways
tracepath -n <target>needs no privileges and gives you the path plus the path MTU in one command. TheResumeline is the answer.- The narrow link is the one leaving the address that reported the pmtu, not the one after it.
- Path MTU discovery depends entirely on ICMP type 3 code 4 reaching the sender. Blanket denying inbound ICMP breaks it silently.
ip route get <dest>shows the cached PMTU, expires in 600 seconds by default, and is the fastest check on the box.ping -M do -s 1472returningsendmsg: Message too longis healthy. The same command returning silent 100 percent loss is a black hole.- On IOS,
show interfacesandshow ip interfacereport different MTUs for the same port. Read the IP one. net.ipv4.tcp_mtu_probing=1works around somebody else's broken firewall, for TCP only.- Small packets fine, large packets hanging, ping clean: assume MTU before anything else.
For the path itself rather than its width, see traceroute in UDP, ICMP and TCP modes, and for watching a path over time see mtr. The DF bit testing technique used above is covered fully in ping, MTU and the DF bit within the ping cluster, and every tool in this series is indexed on the Linux networking commands guide.