> ## Content Index
> Fetch the complete content index at: https://www.pinglabz.com/llms.txt
> Use this file to discover other available public pages before exploring further.

# IPv6 Packet Too Big: Why TCP Uploads Stall and How to Fix It
- URL: https://www.pinglabz.com/ipv6-packet-too-big-tcp-stall-lab/
- Published: 2026-10-13T06:25:24.000Z
- Updated: 2026-10-13T06:25:24.000Z
- Description: Small requests pass while a 64 KiB upload stalls at a 1280-byte link. Restore Packet Too Big feedback and see how a cached PMTU hides the fault.
- Author: Jaime
- Tags: IPv6, Troubleshooting, Packet Analysis, Linux Labs

Three small pings and a small HTTP request passed. A 65,536-byte upload to the same IPv6 server completed its TCP handshake, then timed out. The router's server-facing link had an MTU of 1280, while the client still sent 1500-byte IPv6 packets.

The router was dropping the Packet Too Big messages it generated. Restoring that feedback let the client learn the smaller path MTU, retransmit smaller packets, and finish the upload. A subsequent test with feedback blocked passed anyway because the client had already cached 1280\. We'll reproduce both the failure and that misleading repeat.

## A smaller link beyond the client's first hop

This separate lab uses three namespaces with no uplink:

- `plmtu-client`: `eth0`, `2001:db8:71::10/64`, gateway `2001:db8:71::1`.
- `plmtu-router`: `lan0`, `2001:db8:71::1/64`; `wan0`, `2001:db8:72::1/64`.
- `plmtu-server`: `eth0`, `2001:db8:72::20/64`, gateway `2001:db8:72::1`.

All interfaces begin at 1500\. Only router `wan0` is later reduced to 1280\. The endpoint interfaces stay at 1500, so their initial TCP MSS advertisements do not reveal this downstream limit. Both SYN directions advertised an MSS of 1440 in the baseline capture. We inspect the IP packet size separately from MSS and Ethernet frame length.

Tested with: Ubuntu 26.04.1 LTS, kernel `7.0.0-38-generic`, iproute2 `6.19.0`, nftables `1.1.6`, tcpdump `4.99.6`, Python `3.14.4`, TShark `4.6.4`. You need sudo and those tools; ethtool is used to disable offload aggregation on the isolated veth capture points. The [namespace foundation](https://www.pinglabz.com/linux-namespace-network-lab/) explains the stacks.

## Create an inspectable IPv6 path

In a fresh directory, create `scripts/` and `evidence/`. Save this as `scripts/setup-mtu.sh`, then run it as root with `sudo bash scripts/setup-mtu.sh`:

```bash
#!/usr/bin/env bash
set -euo pipefail
for ns in plmtu-client plmtu-router plmtu-server; do
  if ip netns list | awk '{print $1}' | grep -qx "$ns"; then
    echo "Existing namespace: $ns; stop before reusing it" >&2; exit 1
  fi
done
for dev in mtu-c mtu-l mtu-w mtu-s; do
  if ip link show "$dev" >/dev/null 2>&1; then
    echo "Existing interface: $dev; stop before reusing it" >&2; exit 1
  fi
done
for ns in plmtu-client plmtu-router plmtu-server; do
  ip netns add "$ns"
  ip -n "$ns" link set lo up
done
ip link add mtu-c type veth peer name mtu-l
ip link add mtu-w type veth peer name mtu-s
ip link set mtu-c netns plmtu-client
ip link set mtu-l netns plmtu-router
ip link set mtu-w netns plmtu-router
ip link set mtu-s netns plmtu-server
ip -n plmtu-client link set mtu-c name eth0
ip -n plmtu-router link set mtu-l name lan0
ip -n plmtu-router link set mtu-w name wan0
ip -n plmtu-server link set mtu-s name eth0
ip -n plmtu-client -6 addr add 2001:db8:71::10/64 dev eth0 nodad
ip -n plmtu-router -6 addr add 2001:db8:71::1/64 dev lan0 nodad
ip -n plmtu-router -6 addr add 2001:db8:72::1/64 dev wan0 nodad
ip -n plmtu-server -6 addr add 2001:db8:72::20/64 dev eth0 nodad
ip -n plmtu-client link set eth0 up
ip -n plmtu-router link set lan0 up
ip -n plmtu-router link set wan0 up
ip -n plmtu-server link set eth0 up
ip -n plmtu-client -6 route add default via 2001:db8:71::1
ip -n plmtu-server -6 route add default via 2001:db8:72::1
ip netns exec plmtu-router sysctl -w net.ipv6.conf.all.forwarding=1
# Make the packet representation inspectable at these isolated veth points.
for pair in 'plmtu-client eth0' 'plmtu-router lan0' 'plmtu-router wan0' 'plmtu-server eth0'; do
  read -r ns dev <<< "$pair"
  ip netns exec "$ns" ethtool -K "$dev" tso off gso off gro off
done
# Record and pin the sender behavior for this bounded black-hole comparison.
ip netns exec plmtu-client sysctl -w net.ipv4.tcp_mtu_probing=0
```

The offload changes apply only to these new veth interfaces. The client explicitly uses `net.ipv4.tcp_mtu_probing=0`; despite that sysctl's name, it controls the shared TCP packetization-layer probing setting used for this test. The [Linux sysctl reference](https://www.kernel.org/doc/html/v7.0/networking/ip-sysctl.html?ref=pinglabz.com) describes the modes. Pinning and recording it keeps a different recovery mechanism from changing the bounded comparison.

## Prove delivery with an application checksum

Save this as `scripts/http-mtu.py`:

```python
#!/usr/bin/env python3
from http.server import BaseHTTPRequestHandler,ThreadingHTTPServer
import hashlib,socket
class Handler(BaseHTTPRequestHandler):
    def do_GET(self):
        self.send_response(200);self.send_header('Content-Length','3');self.end_headers();self.wfile.write(b'OK\n')
    def do_POST(self):
        self.connection.settimeout(10)
        size=int(self.headers['Content-Length'])
        try:data=self.rfile.read(size)
        except (TimeoutError,ConnectionResetError) as e:
            print('incomplete upload',self.client_address[1],type(e).__name__,flush=True);return
        print('received',len(data),'of',size,'sha256',hashlib.sha256(data).hexdigest(),flush=True)
        if len(data)!=size:return
        body=(str(len(data))+' '+hashlib.sha256(data).hexdigest()+'\n').encode()
        self.send_response(200);self.send_header('Content-Length',str(len(body)));self.end_headers();self.wfile.write(body)
class Server(ThreadingHTTPServer):address_family=socket.AF_INET6
Server(('2001:db8:72::20',8080),Handler).serve_forever()
```

Start the service in terminal A:

```bash
sudo ip netns exec plmtu-server python3 -u scripts/http-mtu.py
```

It returns `OK` for a small GET. For an upload it reads the declared body length and returns the byte count and SHA-256\. This distinguishes application delivery from data handed to the client's socket.

Save the client as `scripts/client-mtu.py`:

```python
#!/usr/bin/env python3
import hashlib,http.client,json,sys,time,socket,struct
port=int(sys.argv[1]);mode=sys.argv[2];start=time.monotonic()
c=http.client.HTTPConnection('2001:db8:72::20',8080,timeout=5,source_address=('2001:db8:71::10',port))
body=b'x'*65536
failed=False
try:
    if mode=='small':c.request('GET','/small')
    else:c.request('POST','/upload',body=body,headers={'Content-Type':'application/octet-stream'})
    r=c.getresponse();result=r.read().decode()
    expected='OK\n' if mode=='small' else str(len(body))+' '+hashlib.sha256(body).hexdigest()+'\n'
    assert r.status==200 and result==expected
    print(json.dumps({'mode':mode,'port':port,'status':r.status,'result':result.strip(),'seconds':time.monotonic()-start}))
except Exception as e:
    failed=True
    print(json.dumps({'mode':mode,'port':port,'error':type(e).__name__,'message':str(e),'seconds':time.monotonic()-start}));sys.exit(1)
finally:
    if failed and c.sock is not None:
        # Bound this disposable test: discard queued upload bytes after timeout.
        c.sock.setsockopt(socket.SOL_SOCKET,socket.SO_LINGER,struct.pack('ii',1,0))
    c.close()
```

The five-second timeout bounds each request. On failure, this disposable client discards its remaining queued bytes with an abortive close. An earlier version used an ordinary close; its timed-out upload continued in the kernel and reached the server after feedback was restored. The saved run was repeated with this client so the recovery evidence belongs to a new test tuple.

Run the baseline from terminal B:

```bash
sudo ip netns exec plmtu-server ss -lntp 'sport = :8080'
sudo ip netns exec plmtu-client ip -6 route get 2001:db8:72::20
sudo ip netns exec plmtu-client python3 scripts/client-mtu.py 43011 upload
```

The tested response was HTTP 200 with:

```text
65536 1f8745f0d2d1387ec1af2211a3cf417b2e9e885e853472649c1d979d0e9370e3
```

The client checked that result against the body it generated.

## Block only the router's Packet Too Big feedback

Lower the one outgoing link:

```bash
sudo ip -n plmtu-router link set wan0 mtu 1280
sudo ip -n plmtu-router -br link
```

Save this as `scripts/mtu-feedback.nft`:

```nft
table ip6 mtu_feedback {
 chain output { type filter hook output priority 0; policy accept;
  oifname "lan0" ip6 daddr 2001:db8:71::10 icmpv6 type packet-too-big counter drop comment "lab-block-ptb"
 }
}
```

Load the rule in the router namespace before sending an oversized packet:

```bash
sudo ip netns exec plmtu-router nft -c -f scripts/mtu-feedback.nft
sudo ip netns exec plmtu-router nft -f scripts/mtu-feedback.nft
sudo ip netns exec plmtu-client ping -6 -n -c 3 2001:db8:72::20
sudo ip netns exec plmtu-client python3 scripts/client-mtu.py 43012 small
```

All three small probes replied, and the GET returned HTTP 200 with `OK`. Neither result requires a 1500-byte packet to cross the reduced link.

The deliberately selected drop is in `output`: the router creates this error itself. It is not a forwarded client TCP packet. The rule targets type 2 feedback to this client on `lan0`; it does not block Neighbor Discovery or every ICMPv6 message. The [nftables trace lab](https://www.pinglabz.com/nftables-monitor-trace-packet-drop/) explains hook selection.

## Locate the oversized packets on both router sides

Before the upload, start these captures in separate terminals on the Linux host:

```bash
sudo ip netns exec plmtu-router tcpdump --immediate-mode -U -nn -i lan0 -w evidence/blocked-lan0.pcap 'ip6'
```

```bash
sudo ip netns exec plmtu-router tcpdump --immediate-mode -U -nn -i wan0 -w evidence/blocked-wan0.pcap 'ip6'
```

Wait for the listening messages, then run:

```bash
sudo ip netns exec plmtu-client python3 scripts/client-mtu.py 43013 upload
sudo ip netns exec plmtu-router nft list table ip6 mtu_feedback
sudo ip netns exec plmtu-client ip -6 route get 2001:db8:72::20
```

The upload timed out without receiving HTTP 200\. Stop captures with Ctrl+C and keep their final counts. The saved repeat showed the handshake on both sides. Client-to-server TCP data in 1500-byte IPv6 packets appeared at `lan0` but did not cross `wan0`. The feedback rule dropped 15 generated messages in the saved repeat; no Packet Too Big reached the client, and its route lookup did not yet report `mtu 1280`.

Use the selected source port to isolate this attempt. For example, inspect TCP data at ingress:

```bash
tshark -r - -Y 'tcp.srcport == 43013 && tcp.len > 0 && !icmpv6' -T fields -e frame.number -e ipv6.plen -e tcp.seq_raw -e tcp.len < evidence/blocked-lan0.pcap
```

Repeat against `blocked-wan0.pcap`. `ipv6.plen` excludes the 40-byte IPv6 base header. In the measured full-sized packets, 1460 plus 40 was 1500\. This capture has no IPv6 extension headers; that simplifies the TCP/header comparison. Ethernet adds its own bytes and is not included in the IP MTU.

The saved captures had matching captured/filter-received counts and zero kernel drops. Their absence evidence is also supported by the known limiting interface and selected feedback-drop counter. On an unknown path, start by checking capture visibility and direction rather than diagnosing every upload timeout as PMTU failure.

## Restore feedback without enlarging the link

Remove the deliberately introduced rule by emptying only this lab chain:

```bash
sudo ip netns exec plmtu-router nft flush chain ip6 mtu_feedback output
sudo ip -n plmtu-router link show wan0
```

Its MTU stays at 1280\. Restart the two captures under new filenames, such as `recovery-lan0.pcap` and `recovery-wan0.pcap`, then use a new tuple:

```bash
sudo ip netns exec plmtu-client python3 scripts/client-mtu.py 43014 upload
sudo ip netns exec plmtu-client ip -6 route get 2001:db8:72::20
```

The recovery returned HTTP 200 with the same 65,536-byte count and SHA-256\. The route lookup now contained:

```text
expires 599sec mtu 1280
```

Inspect the feedback separately from the quoted original packet inside it:

```bash
tshark -r - -Y 'icmpv6.type == 2' -T fields -e frame.number -e ipv6.src -e ipv6.dst -e icmpv6.mtu < evidence/recovery-lan0.pcap
```

The recovery ingress capture contained ten type 2 messages advertising 1280 to the client. Subsequent upload data crossed `wan0` in IPv6 packets no larger than 1280\. The original oversized packets still appear in the ingress capture before adaptation, so an ingress maximum of 1500 is not evidence that adaptation failed. Use time, tuple, and TCP sequence ranges to follow the change.

[RFC 8201](https://www.rfc-editor.org/rfc/rfc8201.html?ref=pinglabz.com) describes this feedback-driven IPv6 discovery: a router unable to forward an oversized packet returns Packet Too Big, and the source reduces its path estimate. IPv6 routers do not fragment forwarded packets. The successful smaller-packet transfer here supplies the endpoint evidence in addition to the error message.

## A cached PMTU can hide the fault on a repeat

Keep the discovered client state and briefly reintroduce the same lab drop:

```bash
sudo ip netns exec plmtu-router nft add rule ip6 mtu_feedback output oifname lan0 ip6 daddr 2001:db8:71::10 icmpv6 type packet-too-big counter drop
sudo ip netns exec plmtu-client python3 scripts/client-mtu.py 43015 upload
sudo ip netns exec plmtu-router nft list table ip6 mtu_feedback
sudo ip netns exec plmtu-router nft flush chain ip6 mtu_feedback output
```

This upload also returned HTTP 200\. The client started with the cached 1280 estimate, its packets fit, and the newly added drop counter stayed at zero. It did not need new feedback. Changing source ports alone did not clear the destination's PMTU state.

Record `ip -6 route get` before each comparison. To repeat the original cold-state test in this isolated client namespace, stop old requests, restore the 1500 baseline link, and reset its cached route state before repeating the sequence:

```bash
sudo ip -n plmtu-router link set wan0 mtu 1500
sudo ip netns exec plmtu-client ip -6 route flush cache
sudo ip netns exec plmtu-client ip -6 route get 2001:db8:72::20
```

The saved clean repeat used that scoped reset, and its pre-test lookup no longer contained the learned MTU. Do not apply a cache reset to the host's normal network to reproduce this experiment. Stop the foreground HTTP service with Ctrl+C when done.

For the surrounding tools, see [tracepath and PMTU](https://www.pinglabz.com/tracepath-pmtu/) and the [IPv4 ping/DF investigation](https://www.pinglabz.com/ping-mtu-df-bit/). For ingress packets rejected before local delivery, use [the reverse-path filtering lab](https://www.pinglabz.com/linux-rp-filter-asymmetric-routing-lab/). Here the diagnostic sequence is different: transport opens, small requests work, oversized packets stop at a smaller outgoing link, and restored feedback changes the sender's packet size. Endpoint probing and cached state can alter the outcome, so keep both in the test record.