ethtool: Link Speed, Duplex, Offloads, NIC Stats

When a Linux host is slow and nobody can say why, the argument usually stalls at the same place: is it the network, or is it the box. ip will not settle it. ip stops at the kernel's generic view of an interface, and everything interesting below that line (negotiated speed, duplex, which work the NIC is doing on the CPU's behalf, whether the ring buffers are overflowing) belongs to the driver. ethtool is how you ask the driver.

This article covers the flags that earn their place: link settings, driver identification, offloads, per-queue statistics, ring buffers, coalescing and channels. It also does something most ethtool articles avoid, which is being straight about what a virtual NIC can and cannot tell you. Everything below is real output from a VMware vmxnet3 interface on a Debian 13 host bridged into a Cisco Modeling Labs topology. Where a field reads n/a, that is the honest answer for paravirtual hardware, and saying so is more useful than pretending otherwise. It is part of the Linux networking commands cluster.

Link settings: speed, duplex, autonegotiation

The command with no flags shows the link itself:

j@llmbits:~$ sudo ethtool ens224
Settings for ens224:
	Supported ports: [ TP ]
	Supported link modes:   1000baseT/Full
	                        10000baseT/Full
	Supported pause frame use: No
	Supports auto-negotiation: No
	Supported FEC modes: Not reported
	Advertised link modes:  Not reported
	Advertised pause frame use: No
	Advertised auto-negotiation: No
	Advertised FEC modes: Not reported
	Speed: 10000Mb/s
	Duplex: Full
	Auto-negotiation: off
	Port: Twisted Pair
	PHYAD: 0
	Transceiver: internal
	MDI-X: Unknown
	Supports Wake-on: uag
	Wake-on: d
	Link detected: yes

On physical hardware this is the first thing you check on a suspected performance problem, and there are three fields that matter more than the rest.

Link detected is carrier. If this says no, stop reading the rest of the output and go look at the cable, the SFP or the switch port. It corresponds to LOWER_UP in ip link show, and it is the difference between "configured" and "connected".

Speed and Duplex are what actually got negotiated, not what you configured. A gigabit port reporting 100Mb/s is usually a cable or an SFP problem. A port reporting Half duplex on anything modern means autonegotiation failed and one side fell back, which produces late collisions, retransmits and a host that feels broken without any interface reporting errors. This is the single highest-value check ethtool offers on real hardware.

Auto-negotiation tells you whether the two ends agreed or whether somebody hard-set one side. Hard-setting one side and leaving the other on auto is the classic way to produce a duplex mismatch, and it still happens.

Now the honest part. This interface is a paravirtual NIC, and its output says so if you read it. Supports auto-negotiation: No, Auto-negotiation: off, and Speed: 10000Mb/s that was never negotiated with anything. There is no PHY, no cable and no link partner. The hypervisor reports a number the driver passes along.

The router at the other end of this segment reports its own invented number for the same link:

R1#show interfaces Ethernet0/0 | include line protocol|Hardware|Internet|MTU
Ethernet0/0 is up, line protocol is up
  Hardware is AmdP2, address is aabb.cc00.0400 (bia aabb.cc00.0400)
  Internet address is 10.77.0.1/24
  MTU 1500 bytes, BW 10000 Kbit/sec, DLY 1000 usec,

Ten thousand megabits per second on one side and ten thousand kilobits per second on the other, for the same virtual wire. Two emulators, two defaults, neither measured. Anything you actually want to know about throughput on a virtual link has to come from a test, which is what the iPerf cluster is for. Link detected: yes is still meaningful here (it tracks whether the virtual NIC is connected to a virtual switch), and everything about offloads, rings and counters further down is entirely real.

Which driver, which slot

j@llmbits:~$ sudo ethtool -i ens224
driver: vmxnet3
version: 1.9.0.0-k-NAPI
firmware-version:
expansion-rom-version:
bus-info: 0000:13:00.0
supports-statistics: yes
supports-test: no
supports-eeprom-access: no
supports-register-dump: yes
supports-priv-flags: no

The first command to run when you are told "the NIC is misbehaving", because it tells you what you are actually dealing with. driver and version go straight into a search for known bugs. bus-info is the PCI address, which is how you correlate an interface name with a physical slot, or with a card in lspci. On a box with six identical NICs and an unhelpful naming scheme, this is how you work out which one has the amber light.

firmware-version being empty is expected here; on real hardware it is one of the first things a vendor support case will ask for. The supports-* lines tell you which of ethtool's other functions this driver actually implements, which saves you interpreting a missing feature as a broken card. supports-test: no here means ethtool -t will not run a self test, and that is a driver limitation rather than a fault.

Offloads: what the NIC is doing instead of your CPU

j@llmbits:~$ sudo ethtool -k ens224
Features for ens224:
rx-checksumming: on
tx-checksumming: on
	tx-checksum-ipv4: off [fixed]
	tx-checksum-ip-generic: on
	tx-checksum-ipv6: off [fixed]
	tx-checksum-fcoe-crc: off [fixed]
	tx-checksum-sctp: off [fixed]
scatter-gather: on
	tx-scatter-gather: on
	tx-scatter-gather-fraglist: off [fixed]
tcp-segmentation-offload: on
	tx-tcp-segmentation: on
	tx-tcp-ecn-segmentation: off [fixed]
	tx-tcp-mangleid-segmentation: off
	tx-tcp6-segmentation: on
generic-segmentation-offload: on
generic-receive-offload: on
large-receive-offload: on
rx-vlan-offload: on
tx-vlan-offload: on
ntuple-filters: off [fixed]
receive-hashing: on
highdma: on
rx-vlan-filter: on [fixed]
vlan-challenged: off [fixed]
tx-gso-robust: off [fixed]
tx-fcoe-segmentation: off [fixed]
tx-gre-segmentation: off [fixed]
tx-gre-csum-segmentation: off [fixed]

[fixed] means the driver will not let you change it. Everything else is toggleable with ethtool -K.

The four that matter to a network engineer:

Three consequences you will actually run into.

First, packet captures lie when offloads are on. tcpdump taps the stack above the NIC, so on transmit it sees the oversized pre-segmentation buffer and on receive it sees the merged GRO buffer. You will see "packets" of 30 kilobytes on a 1500 byte MTU link and checksums marked incorrect on outbound traffic, because the NIC had not filled them in yet. Neither is a fault. If you need the capture to reflect the wire, turn the offloads off for the duration: sudo ethtool -K ens224 tso off gso off gro off lro off, capture, then turn them back on. This is the single most common reason a capture disagrees with a switch counter, and it comes up constantly in tcpdump work.

Second, LRO breaks forwarding. If the host routes or bridges (a router, a firewall, a container host), LRO merges frames and discards information needed to reconstruct them correctly on the way out. The kernel disables it automatically when forwarding is enabled, but on a box where forwarding gets turned on later, check it.

Third, the byte counters will not match. This is where the per-queue statistics become interesting.

Per-queue statistics, and the TSO proof

ethtool -S dumps driver-level counters, which are far more detailed than the kernel's generic set. On this NIC that is six transmit queues and six receive queues. An excerpt:

j@llmbits:~$ sudo ethtool -S ens224
NIC statistics:
     Tx Queue#: 0
       TSO pkts tx: 0
       TSO bytes tx: 0
       ucast pkts tx: 3
       ucast bytes tx: 222
       mcast pkts tx: 1
       mcast bytes tx: 669
       bcast pkts tx: 5
       bcast bytes tx: 1635
       pkts tx err: 0
       pkts tx discard: 0
       drv dropped tx total: 0
          too many frags: 0
          giant hdr: 0
          hdr err: 0
          tso: 0
       ring full: 0
       pkts linearized: 0
     Tx Queue#: 1
       TSO pkts tx: 2532
       TSO bytes tx: 8251296
       ucast pkts tx: 3247
       ucast bytes tx: 9317987
       mcast pkts tx: 5
       mcast bytes tx: 490
       bcast pkts tx: 0
       bcast bytes tx: 0
       pkts tx err: 0
       pkts tx discard: 0
       drv dropped tx total: 0
       ring full: 0
       pkts linearized: 0
     ...
     Rx Queue#: 2
       LRO pkts rx: 0
       LRO byte rx: 0
       ucast pkts rx: 3640
       ucast bytes rx: 244942
       mcast pkts rx: 0
       mcast bytes rx: 0
       pkts rx OOB: 0
       pkts rx err: 0
       drv dropped rx total: 0
          err: 0
          fcs: 0
       rx buf alloc fail: 0
     ...
     tx timeout count: 0

Read Tx Queue 1. 2532 TSO packets carrying 8,251,296 bytes. That is an average of 3258 bytes per "packet" on a link with a 1500 byte MTU, and it is the offload working exactly as designed: the kernel handed the NIC 2532 large buffers and the NIC cut them into MTU-sized frames on the way out. The switch on the other end counted several times as many frames as the host counted packets, and neither counter is wrong.

That is worth internalizing, because comparing a host's packet count against a switch port's packet count is a diagnostic people reach for and it does not mean what they think it means whenever TSO is on.

Two other patterns to look for in this output. Uneven queue distribution is visible immediately here: nearly all traffic is on Tx Queue 1 and Rx Queue 2 while the others are idle. On a lightly loaded host that is just flow hashing with very few flows. On a busy host, one saturated queue while five sit idle means the receive hash is not spreading your traffic, which is a real and fixable throughput ceiling. ring full, rx buf alloc fail and pkts rx OOB are the counters that mean the host could not keep up with the NIC, which is a different problem from the network dropping frames and is fixed in a different place.

Counter names are driver-specific, which is the main annoyance with ethtool -S. An Intel ixgbe card and this vmxnet3 interface name similar concepts differently, so a monitoring check built against one driver will silently collect nothing on another.

Ring buffers

j@llmbits:~$ sudo ethtool -g ens224
Ring parameters for ens224:
Pre-set maximums:
RX:			4096
RX Mini:		2048
RX Jumbo:		4096
TX:			4096
Current hardware settings:
RX:			1024
RX Mini:		128
RX Jumbo:		512
TX:			512

The ring is the queue of descriptors between the NIC and the driver. When the NIC receives faster than the kernel drains, the ring fills and frames are dropped, and those drops appear as rx_no_buffer, rx_missed_errors or the driver's local equivalent, plus the missed column in ip -s link.

Here RX is at 1024 out of a possible 4096 and TX at 512 out of 4096, which is typical: drivers ship conservative defaults. Raising them with ethtool -G ens224 rx 4096 is the standard first move on a host dropping frames under burst, and it costs a small amount of memory and a small amount of latency (a deeper queue means a packet can wait longer). Raise it if you are dropping, not preemptively.

Worth knowing: on most drivers, changing the ring size resets the interface. That is a brief outage on a production link, so schedule it rather than discovering it.

Coalescing and channels

j@llmbits:~$ sudo ethtool -c ens224
Coalesce parameters for ens224:
Adaptive RX: off  TX: n/a
rx-usecs:	250
rx-frames:	0
tx-usecs:	n/a
tx-frames:	0
CQE mode RX: n/a  TX: n/a

Interrupt coalescing is the throughput against latency dial. rx-usecs: 250 means the NIC waits up to 250 microseconds to batch received frames before raising an interrupt. Fewer interrupts means less CPU and more throughput; the cost is up to 250 microseconds of added latency. For bulk transfer that is a good trade. For a latency-sensitive workload it is not, and dropping rx-usecs is the knob. Most of the fields here read n/a because vmxnet3 implements only a subset, which is normal and driver-dependent.

j@llmbits:~$ sudo ethtool -l ens224
Channel parameters for ens224:
Pre-set maximums:
RX:		n/a
TX:		n/a
Other:		1
Combined:	6
Current hardware settings:
RX:		n/a
TX:		n/a
Other:		1
Combined:	6

Six combined channels, each a queue pair with its own interrupt, which matches the twelve queues in the statistics output. Combined channels are already at maximum here. On hardware where they are not, raising them lets you spread interrupt load across more CPU cores, which matters once a single core handling softirqs becomes the bottleneck.

And a case of a feature genuinely not being there:

j@llmbits:~$ sudo ethtool -a ens224
j@llmbits:~$

No output at all. vmxnet3 does not implement the pause-parameter call, which is consistent with Supported pause frame use: No in the link settings. On a physical NIC this is where you would check whether 802.3x flow control is enabled, which matters a great deal on a converged fabric and is usually something you want off on a modern network in favor of proper queueing.

A troubleshooting order

When a host has a network performance problem, run these in this sequence:

None of these settings persist. ethtool writes to the running driver and a reboot or a driver reload undoes it. Persisting a change means an ethtool-opts line under /etc/network/interfaces, a dispatcher.d script under NetworkManager, or a udev rule. That is the same lesson as the ip command: the live tool is for testing, and something else owns persistence.

FAQ

Why does ethtool show "Speed: Unknown!" or a nonsense value?

Either there is no carrier, in which case there is nothing to report and Link detected will say no, or the interface is virtual and the driver returns whatever the hypervisor told it. On a bond or a bridge, the value is synthesized from the members and is not a negotiated speed at all.

Should I turn offloads off?

Not permanently. They are a large CPU saving and modern drivers implement them well. Turn TSO, GSO, GRO and LRO off temporarily when you need a packet capture that matches the wire, and turn them back on afterward. The exception is LRO on a host that forwards traffic, where it should stay off, and the kernel usually handles that for you.

What does [fixed] mean in the ethtool -k output?

The driver does not permit that feature to be changed at runtime, either because the hardware cannot do it or because the driver has not implemented the toggle. ethtool -K will refuse. It is not an error state.

Do ethtool settings survive a reboot?

No. Ring sizes, coalescing, channels and offload toggles all reset when the driver reloads. Persist them in an ethtool-opts line in /etc/network/interfaces, a NetworkManager dispatcher script, or a udev rule, depending on the distribution.

Is ethtool useful on a virtual machine at all?

Yes, with the speed and duplex fields discounted. Driver identification, offload state, ring sizes, per-queue statistics and drop counters are all real and all actionable on a VM, because the paravirtual driver is real software doing real work. What you cannot learn from a VM is anything about the physical link, which lives on the hypervisor.

Key takeaways

• ethtool is where the kernel's generic view ends and the driver's begins. ip cannot tell you speed, duplex, offloads or ring depth.
• Link detected, Speed and Duplex are the three fields that resolve most physical-layer arguments. Half duplex on modern gear means autonegotiation failed.
• On a virtual NIC, speed and duplex are numbers the hypervisor made up. Everything about offloads, rings, queues and drops is still real.
• TSO means the host counts far fewer, far larger "packets" than the wire carries. In this capture, 2532 TSO packets carried 8.25 MB on a 1500 byte MTU link.
• Packet captures do not match the wire while offloads are on. Turn TSO, GSO, GRO and LRO off for the duration of a capture, then turn them back on.
• LRO must stay off on any host that forwards or bridges traffic.
• ethtool -S is where host-side drops show up: ring full, rx buf alloc fail, pkts rx OOB. Those are a different problem from network loss.
• Nothing ethtool sets persists across a reboot or a driver reload.

Next, make an interface configuration survive that reboot with nmcli and nmtui. For the layer above this one, the Linux ip command guide covers the kernel's view, and everything in this cluster is indexed on the Linux networking commands guide.