> ## Content Index > Fetch the complete content index at: https://www.pinglabz.com/llms.txt > Use this file to discover other available public pages before exploring further. # iPerf3 Parallel Streams and TCP Window Size (-P and -w) - URL: https://www.pinglabz.com/iperf3-parallel-streams-tcp-window/ - Published: 2026-07-09T05:11:42.000Z - Updated: 2026-07-09T05:17:03.000Z - Description: Why fast links benchmark slow: bandwidth-delay product, a real 78% throughput collapse from a 64K window at 104 ms RTT, and how parallel streams buy it back. - Author: Jaime - Tags: iPerf, Networking, #Import 2026-08-01 19:54 Two iPerf3 flags separate people who run throughput tests from people who understand them: `-P` (parallel streams) and `-w` (TCP window size). Together they explain the most common head-scratcher in network testing - a fast link that benchmarks slow the moment latency enters the picture. We reproduced that exact failure on a live Cisco Modeling Labs topology with 100 ms of injected round-trip time, so every number below is real. This article is part of our [complete iPerf guide](https://www.pinglabz.com/iperf/). ## The Bandwidth-Delay Product, Quickly TCP can only keep a window's worth of unacknowledged data in flight. Once the window is full, the sender stops and waits for ACKs, and ACKs take a round trip. So the ceiling on any single TCP flow is: ``` max throughput = window size / round-trip time ``` Flip it around and you get the bandwidth-delay product (BDP), the window you need to fill a given path: ``` BDP = bandwidth x RTT 35 Mbit/sec x 0.104 sec = ~3.6 Mbit = ~455 KB ``` Our lab path carries about 35 Mbit/sec with 104 ms RTT, so a single flow needs roughly 455 KB of window to fill it. Hold that number; you are about to watch what happens when TCP has far less. ## The Experiment: Same Path, Three Windows First, the path with negligible latency (RTT around 3.6 ms). A default iPerf3 run lands at 34.6 Mbit/sec - the path's natural ceiling. Then we added 50 ms of one-way delay on the WAN link (104 ms RTT, confirmed by ping) and ran three tests. ### Default window, 104 ms RTT ``` client:~$ iperf3 -c 10.0.20.10 [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 5.00 MBytes 41.9 Mbits/sec 5 539 KBytes [ 5] 3.00-4.00 sec 2.75 MBytes 23.1 Mbits/sec 0 318 KBytes [ 5] 6.00-7.00 sec 2.75 MBytes 23.0 Mbits/sec 0 321 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 0.00-10.00 sec 31.4 MBytes 26.3 Mbits/sec 39 sender [ 5] 0.00-10.11 sec 27.8 MBytes 23.0 Mbits/sec receiver ``` Linux autotuning grows the window past 300 KB and recovers most of the throughput: 26.3 Mbit/sec. Not bad. Watch the Cwnd column doing exactly what the BDP math predicts it must. ### Forced 64 KB window, same path ``` client:~$ iperf3 -c 10.0.20.10 -w 64K [ ID] Interval Transfer Bitrate Retr Cwnd [ 5] 0.00-1.00 sec 640 KBytes 5.24 Mbits/sec 0 161 KBytes [ 5] 4.00-5.00 sec 768 KBytes 6.29 Mbits/sec 0 112 KBytes - - - - - - - - - - - - - - - - - - - - - - - - - [ 5] 0.00-10.00 sec 6.88 MBytes 5.77 Mbits/sec 11 sender [ 5] 0.00-10.10 sec 6.88 MBytes 5.71 Mbits/sec receiver ``` Same link, same routers, same everything - 5.77 Mbit/sec. The math checks out almost exactly: 64 KB per 104 ms round trip is about 5 Mbit/sec. Nothing is broken. TCP is simply spending most of its time waiting for ACKs. This is what legacy applications with small fixed socket buffers experience on every long-haul path, and it is why "the link is 1 Gbps but the transfer runs at 50 Mbps" is usually not a network problem at all. ### Same tiny window, four parallel streams ``` client:~$ iperf3 -c 10.0.20.10 -w 64K -P 4 [ ID] Interval Transfer Bitrate Retr [ 5] 0.00-10.00 sec 7.00 MBytes 5.87 Mbits/sec 0 sender [ 7] 0.00-10.00 sec 4.50 MBytes 3.77 Mbits/sec 4 sender [ 9] 0.00-10.00 sec 6.88 MBytes 5.77 Mbits/sec 1 sender [ 11] 0.00-10.00 sec 6.88 MBytes 5.77 Mbits/sec 1 sender [SUM] 0.00-10.00 sec 25.2 MBytes 21.2 Mbits/sec 6 sender [SUM] 0.00-10.11 sec 25.2 MBytes 21.0 Mbits/sec receiver ``` Each stream is still window-starved at roughly 5 Mbit/sec, but four of them together deliver 21.2 Mbit/sec. Parallel streams multiply the effective in-flight data - four windows instead of one. This is exactly why browsers open multiple connections and why backup tools have a "streams" setting. ## What the Comparison Tells You **Default window** 26.3 Mbit/sec. Modern autotuning handles moderate BDPs on its own. If your OS is current and buffers are not capped, you rarely need `-w`. **\-w 64K** 5.77 Mbit/sec. A 78% collapse from a setting, not a fault. Window / RTT is a hard ceiling no amount of bandwidth fixes. **\-w 64K -P 4** 21.2 Mbit/sec. Parallelism buys back what small windows lose. If `-P` helps a lot, suspect per-flow limits: windows, per-flow policers, or single-flow load-balancing hashes. ## How to Use -P and -w Diagnostically Run a single stream, then `-P 4`, and compare: - **Single stream is slow, parallel sum is much higher:** a per-flow limit. Check RTT and window sizes (BDP math above), per-flow QoS policers, or an ECMP/port-channel hash pinning one flow to one member link. - **Single and parallel both hit the same ceiling:** a shared limit - the link itself, a shaper on the path, or CPU on either endpoint. Confirm the interface rate on the router while the test runs (`show interfaces` and watch the output rate counters climb). - **Parallel is worse than single:** usually endpoint CPU. iPerf3 runs all streams in one thread until recent versions, so a saturated core caps the sum. Check with `top` during the test; iPerf2 multithreads if you need to rule this out (see [iPerf2 vs iPerf3](https://www.pinglabz.com/iperf2-vs-iperf3/)). Two cautions with `-w`. First, the OS may silently cap what you request (check the "socket buffer size" line iPerf3 prints). Second, what you set with `-w` is a buffer size request on both ends; setting it lower than autotune would reach makes things worse, so use it to reproduce problems, not as a routine "optimization." ## Key Takeaways A single TCP flow can never exceed window divided by RTT, so know your path's BDP before judging a test. Modern autotuning usually gets close to the ceiling; small fixed windows crater throughput in exact proportion to the math, and parallel streams recover it by flying multiple windows at once. Use `-P` as a diagnostic fork in the road: it separates per-flow limits from shared limits in under a minute. When a path still disappoints after this analysis, packet loss is the usual suspect - pick up the trail in [troubleshooting slow throughput with iPerf3](https://www.pinglabz.com/troubleshooting-slow-throughput-iperf3/), or go back to the [iPerf complete guide](https://www.pinglabz.com/iperf/) for the full command reference.