iPerf

Troubleshooting Slow Network Throughput with iPerf3

We broke a healthy lab path three ways on real Cisco routers. Learn the three fingerprints of slow throughput and the iPerf3 tests that expose each.
Troubleshooting slow throughput with iPerf3 feature image showing TCP collapse under loss
In: iPerf, Networking

"The link is fast but the transfer is slow" is the complaint that never dies. iPerf3 can settle it in minutes, but only if you test methodically - throughput problems have exactly three root causes (not enough capacity, too much latency for the window, or packet loss) and each one leaves a distinct fingerprint in iPerf3 output. To show you those fingerprints, we broke a healthy lab path three different ways on real Cisco routers and captured what iPerf3 reported each time. This article is part of our complete iPerf guide.

Step 0: Know Your Baseline

Our lab path (Linux client, two OSPF-routed IOS XE hops, Linux server) healthy:

client:~$ ping -c 5 10.0.20.10
rtt min/avg/max/mdev = 2.948/3.611/5.239/0.833 ms

client:~$ iperf3 -c 10.0.20.10
[  5]   0.00-10.01  sec  41.2 MBytes  34.6 Mbits/sec   83            sender
[  5]   0.00-10.01  sec  40.9 MBytes  34.2 Mbits/sec                  receiver

34.6 Mbit/sec at 3.6 ms RTT. Without a baseline you cannot tell "degraded" from "normal for this path," so record one per important path while things work (the same discipline we preach for ping).

Fingerprint 1: A Bandwidth Bottleneck

We capped the WAN link at 10 Mbit/sec and re-ran the same test:

client:~$ iperf3 -c 10.0.20.10
[ ID] Interval           Transfer     Bitrate         Retr
[  5]   0.00-10.00  sec  13.8 MBytes  11.5 Mbits/sec    0            sender
[  5]   0.00-10.53  sec  11.9 MBytes  9.46 Mbits/sec                  receiver

The signature: throughput sits pinned just under a suspiciously round number, retransmits are modest, and latency is roughly normal. TCP found the shaper and settled beneath it. Confirm by watching the constraining interface while the test runs - on the router mid-test:

R1# show interfaces Ethernet0/1 | include rate
  5 minute input rate 184000 bits/sec, 206 packets/sec
  5 minute output rate 5747000 bits/sec, 471 packets/sec

If the interface (or a policer/shaper on it) tops out while your test does, you have found the bottleneck. The fix is capacity or QoS policy, not tuning.

Fingerprint 2: Latency Plus a Small Window

Next we removed the cap and injected 50 ms of one-way delay instead. Ping confirms the change; loss stays at zero:

client:~$ ping -c 5 10.0.20.10
rtt min/avg/max/mdev = 103.459/103.962/104.532/0.342 ms

client:~$ iperf3 -c 10.0.20.10
[  5]   0.00-10.00  sec  31.4 MBytes  26.3 Mbits/sec   39            sender

client:~$ iperf3 -c 10.0.20.10 -w 64K
[  5]   0.00-10.00  sec  6.88 MBytes  5.77 Mbits/sec   11            sender

The signature: ping shows high but stable RTT with no loss, autotuned TCP gets close-ish to the baseline, and anything with a constrained window craters in exact proportion to window/RTT (64 KB / 104 ms = about 5 Mbit/sec - the math, the demonstration, and the parallel-stream workaround are in parallel streams and TCP window size). If a real application is slow on this path while iPerf3 is fast, suspect the application's own socket buffers.

Fingerprint 3: Packet Loss

Finally the cruel one: same 104 ms RTT, plus 2% packet loss on the WAN link.

client:~$ ping -c 5 10.0.20.10
5 packets transmitted, 5 received, 0% packet loss
rtt min/avg/max/mdev = 103.646/104.013/104.175/0.190 ms

client:~$ iperf3 -c 10.0.20.10
[ ID] Interval           Transfer     Bitrate         Retr  Cwnd
[  5]   0.00-1.00   sec   640 KBytes  5.24 Mbits/sec    5   36.8 KBytes
[  5]   2.00-3.00   sec   128 KBytes  1.05 Mbits/sec    2   18.4 KBytes
[  5]   4.00-5.00   sec  0.00 Bytes  0.00 bits/sec    0   7.07 KBytes
[  5]   8.00-9.00   sec  0.00 Bytes  0.00 bits/sec    1   11.3 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[  5]   0.00-10.00  sec  1.75 MBytes  1.47 Mbits/sec   20            sender
[  5]   0.00-10.11  sec  1.50 MBytes  1.25 Mbits/sec                  receiver

Look closely: a five-packet ping saw nothing wrong, yet TCP collapsed to 1.47 Mbit/sec with intervals that transferred zero bytes. Two percent loss at 100 ms RTT is catastrophic for TCP because every loss halves the congestion window and recovery takes a full round trip. The Cwnd column never escapes double digits.

Now prove the loss and measure it. UDP does what a short ping cannot:

client:~$ iperf3 -c 10.0.20.10 -u -b 20M
[ ID] Interval           Transfer     Bitrate         Jitter    Lost/Total Datagrams
[  5]   0.00-10.00  sec  23.8 MBytes  20.0 Mbits/sec  0.000 ms  0/17266 (0%)  sender
[  5]   0.00-10.10  sec  23.3 MBytes  19.4 Mbits/sec  0.219 ms  366/17266 (2.1%)  receiver

17,266 probes in 10 seconds versus ping's 5: the UDP test nails the loss at 2.1%. From here it is regular loss-hunting: check interface counters hop by hop for errors, drops, and duplex mismatches, and see whether loss follows a specific link (iPerf3 UDP testing covers the technique in depth).

The Decision Tree

1. Pinned under a round number, low Retr

Bandwidth bottleneck. Verify with interface rate counters mid-test. Fix: capacity or QoS.

2. High stable RTT, no loss, window-sensitive

Latency x window ceiling. Verify: -w 64K collapses, -P 4 recovers. Fix: buffers/BDP tuning, parallelism.

3. Retr climbing, Cwnd tiny, zero-byte intervals

Packet loss. Verify with -u at moderate rate and read the receiver loss %. Fix: find the lossy hop (errors, drops, duplex, overloaded QoS queue).

4. Only parallel streams reach the ceiling

Per-flow limit: policer, hash pinning on a port-channel/ECMP, or one saturated CPU core. Compare single vs -P 4 sums.

Habits That Keep You Honest

Test both directions (-R) before blaming the network; asymmetry is a clue, not noise. Run 30 to 60 second tests so slow start and transient dips average out. Watch top on both endpoints, because a saturated CPU produces numbers that look exactly like a network problem. And when you report a result, quote the receiver line with the RTT alongside it - "34 Mbit/sec at 104 ms RTT" is a measurement, "34 Mbit/sec" is a rumor.

Key Takeaways

Slow throughput has three fingerprints and iPerf3 exposes all of them: a clean pin under a round number means a bandwidth cap, high stable RTT with window sensitivity means a BDP ceiling, and climbing retransmits with a collapsing Cwnd mean loss - which a moderate UDP test will quantify precisely even when ping sees 0%. Baseline your paths while they are healthy, test in both directions, and confirm on the router with interface counters. The complete command reference, server setup, and version guidance all live in the iPerf complete guide.

Written by
More from Ping Labz
Great! You’ve successfully signed up.
Welcome back! You've successfully signed in.
You've successfully subscribed to Ping Labz.
Your link has expired.
Success! Check your email for magic link to sign-in.
Success! Your billing info has been updated.
Your billing was not updated.