Every network problem eventually collapses into one question: did the packet leave, did it arrive, and what did it look like when it got there. Routing tables, interface counters and syslog give you the device's opinion of what happened. A capture gives you what actually crossed the wire, byte for byte, with a microsecond timestamp on it. When those two disagree, the capture wins.
This is the tool you reach for when everyone on the bridge call is confident and nobody is right. The app team says the firewall is dropping traffic, the firewall team says nothing is being denied, the server team says the request never arrived. One capture at the correct tap point settles it in ninety seconds, in a way nobody can argue with, because it is not an interpretation. This guide is for engineers who already know what a TCP flag is and want to stop guessing about them.
It covers what capture solves, how it works from the tap point through the kernel and out through BPF, where to put the capture point, the fields worth reading first, a minimum viable Embedded Packet Capture config on IOS XE, why capture filters and display filters are two different languages, the handling rules that keep a pcap from becoming a data breach, and the failure modes that quietly ruin captures. Every output here came off a real lab: a Debian 13 host running tcpdump 4.99.5 on an interface bridged into Cisco Modeling Labs, three iol-xe routers on IOS XE 17.18.2, and a Kali box running nmap 7.99 and Scapy 2.7.01.
What Packet Capture Solves
Most troubleshooting tools report state, and every one of those reports is produced by the same software you are trying to debug, which means all of them can be confidently wrong at exactly the moment you need them right. Here is that failure mode from the lab. R1 was configured with ip http server and says so:
R1# show ip http server status | include status|port
HTTP server status: Enabled
HTTP server port: 80
HTTP secure server status: DisabledThe client disagrees:
$ curl -sv -m 8 http://192.168.99.1/
* Trying 192.168.99.1:80...
* connect to 192.168.99.1 port 80 from 192.168.99.100 port 58908 failed: Connection refusedTwo sources of truth, flatly contradicting each other. The capture is the tiebreaker and it takes two frames:
$ sudo tcpdump -i ens224 -nn -c 2 'tcp port 80'
15:04:19.316294 IP 192.168.99.100.58922 > 192.168.99.1.80: Flags [S], seq 1309506869, win 64240, length 0
15:04:19.320502 IP 192.168.99.1.80 > 192.168.99.100.58922: Flags [R.], seq 0, ack 1309506870, win 0, length 0A SYN answered by a RST in four milliseconds. Not routing, not a firewall drop and not a slow server: that is the signature of "the packet arrived and nothing was listening". The status output was describing a CLI knob, not a running listener, and no amount of show commands gets you there when the software reporting its own health is the broken part.
That is the class of problem capture owns: two components disagree and you need neutral ground; you have to prove a negative, which stays unfalsifiable until someone captures on the receiving interface and shows an empty file; the protocol is behaving legally but unhelpfully (retransmissions, zero windows, a reset mid-session) so nothing logs an error; or somebody is doing something to you, because scans, spoofed sources and ARP poisoning are obvious on the wire and invisible in a config. What it does not solve is anything that never reached the tap point.
How Capture Works: The 10,000-Foot View
Four things have to happen for a frame to reach your file, and every capture problem is one of them failing.
1. The frame has to reach your tap point. On a switched network your NIC receives its own unicast traffic, broadcasts and joined multicast, and nothing else, because the switch sends you nothing else. To see traffic that is not yours you need a port mirror, a physical tap, a capture point on the device already forwarding the traffic, or a position on the path such as the router or firewall itself. Getting this right is most of the job, so know the Cisco port-mirroring options and when to use each one before you need them at 3am.
2. The NIC has to hand the frame up. Promiscuous mode turns off the NIC's own MAC filter so everything on the segment is passed to the kernel, and libpcap sets it for you. Note the ordering: it only helps if the switch was already sending you the frames. On an ordinary access port it mostly gets you a great view of broadcast traffic, which is why "I enabled promiscuous mode and still see nothing" is such a common complaint.
3. The kernel has to copy it to you. On Linux, capture runs through a privileged AF_PACKET socket into a memory-mapped ring buffer. That is why the most common tcpdump failure is not a filter problem at all:
$ ls -l $(which tcpdump)
-rwxr-xr-x 1 root root 1273880 Feb 9 2025 /usr/bin/tcpdump
$ tcpdump -i ens224 -c 1
tcpdump: ens224: You don't have permission to perform this capture on that device
(socket: Operation not permitted)No setuid bit and no cap_net_raw file capability, so the socket cannot be opened. Fix it with sudo, or grant the binary cap_net_raw,cap_net_admin so named users can capture without full root. Confirm you are pointed at the right NIC while you are there:
$ tcpdump -D
1.ens192 [Up, Running, Connected]
2.ens224 [Up, Running, Connected]
3.any (Pseudo-device that captures on all interfaces) [Up, Running]
4.lo [Up, Running, Loopback]Here ens224 is bridged into the lab topology and ens192 is the management LAN. The wrong one produces a file full of perfectly valid traffic that has nothing to do with your problem.
4. The filter has to run before the copy. BPF expressions are not evaluated by tcpdump in userspace: they are compiled to bytecode and pushed into the kernel, where they run against each frame before it is copied into the ring buffer. A frame that fails the filter is never copied and costs almost nothing. Watch that working in the summary tcpdump prints on exit, which holds three numbers people habitually read as one:
$ sudo tcpdump -i ens224 -nn -c 8
listening on ens224, link-type EN10MB (Ethernet), snapshot length 262144 bytes
14:56:14.914958 IP 0.0.0.0.68 > 255.255.255.255.67: BOOTP/DHCP, Request from 00:0c:29:b1:cc:47, length 301
14:56:16.913287 IP 192.168.99.1 > 224.0.0.5: OSPFv2, Hello, length 84
14:56:18.903784 IP 192.168.99.3 > 224.0.0.5: OSPFv2, Hello, length 84
14:56:19.218986 IP 192.168.99.2 > 224.0.0.5: OSPFv2, Hello, length 84
14:56:26.388141 IP 192.168.99.1 > 224.0.0.5: OSPFv2, Hello, length 84
14:56:28.132532 IP 192.168.99.3 > 224.0.0.5: OSPFv2, Hello, length 84
14:56:28.891504 IP 192.168.99.2 > 224.0.0.5: OSPFv2, Hello, length 84
14:56:35.461351 IP 192.168.99.1 > 224.0.0.5: OSPFv2, Hello, length 84
8 packets captured
8 packets received by filter
0 packets dropped by kernel-c limit or lost some.Check that third number first on any capture you intend to trust. Zero drops means the file is a complete record of everything matching your filter; anything higher means the wire beat your capture process and the missing frames are, by Murphy's law, the interesting ones. Note also that nobody asked for the traffic above: that is R1, R2 and R3 multicasting OSPF Hellos to 224.0.0.5 every ten seconds, which is what a quiet segment actually looks like (the five OSPF packet types and their job on the wire covers what those frames are doing). Control-plane chatter is the first thing you filter out.
Where to Capture: Host, Router, or Firewall
You have three realistic places to put a capture point and they are not interchangeable. Choose by what you need to see and how much you are allowed to disturb.
The Linux host is where you start: it is free, it has the richest filter language, and the same BPF syntax turns up in Wireshark, in Scapy and in half the security tooling you will ever touch. If you only get good at one capture tool, make it this one, and how to actually use tcpdump on a production box is the workhorse article for this cluster.
The Cisco device itself is the answer when the traffic never touches a host you control, or when you need to know what the router saw rather than what the wire carried. Embedded Packet Capture puts a capture point on any interface with no tap, no SPAN session and no laptop in the rack, at the cost of a buffer that lives in device RAM. Detail is in capturing packets on a router with no external sniffer. Even where that is impractical the device can be a sensor: an ACL with log on its permit entries makes a crude detection engine. After a 200-port nmap SYN scan crossed R2:
R2# show ip access-lists SCAN-DETECT
Extended IP access list SCAN-DETECT
10 permit tcp any any eq 22 log (257 matches)
20 permit tcp any any eq telnet log (2 matches)
30 permit tcp any any eq www log (2 matches)
40 permit tcp any any eq 443 log (1 match)
50 permit tcp any host 192.168.99.2 log (1420 matches)
60 permit udp any any log (26 matches)
70 permit icmp any any log (10 matches)
80 permit ip any any (17 matches)Line 50 absorbed 1420 packets aimed at a single host while every other line stayed in double digits. That lopsided distribution against one destination is the shape of a port scan, visible without storing a single packet. IOS rate-limits ACL logging by default though, so syslog alone undercounts, which is the argument for flow records on top of ACL logging when you need volume as well as first-occurrence detection.
The firewall is the right tap point for anything involving translation, policy or inspection, because it shows the same flow on both interfaces with pre-NAT and post-NAT addresses. No host-based capture produces that view. Running a capture on an ASA and reading the result covers the interface-and-ACL model, and simulating a flow through the policy without generating real traffic is the companion trick when you want the firewall's decision before committing to a capture.
Reading a Capture: The Fields That Actually Matter
A decoded frame has hundreds of fields. In practice you read about eight, in a fixed order, and the rest are there for when those eight raise a question.
Timestamps, read as deltas. Round trip time, retransmission timers and the gap between request and response all live in the differences. A three-second pause before a SYN retransmission is a story; the wall clock only matters for correlating with a ticket.
Layer 2, when you suspect layer 2. tcpdump hides the Ethernet header unless you ask with -e. Here is an ARP resolution and the ICMP exchange it unblocked, captured after flushing the neighbour entry:
$ sudo tcpdump -i ens224 -nn -e -c 6 'arp or icmp'
14:56:49.319348 00:0c:29:b1:cc:47 > ff:ff:ff:ff:ff:ff, ethertype ARP (0x0806), length 42: Request who-has 192.168.99.2 tell 192.168.99.100, length 28
14:56:49.323275 aa:bb:cc:00:dc:00 > 00:0c:29:b1:cc:47, ethertype ARP (0x0806), length 60: Reply 192.168.99.2 is-at aa:bb:cc:00:dc:00, length 46
14:56:49.323306 00:0c:29:b1:cc:47 > aa:bb:cc:00:dc:00, ethertype IPv4 (0x0800), length 98: 192.168.99.100 > 192.168.99.2: ICMP echo request, id 6, seq 1, length 64
14:56:49.325668 aa:bb:cc:00:dc:00 > 00:0c:29:b1:cc:47, ethertype IPv4 (0x0800), length 98: 192.168.99.2 > 192.168.99.100: ICMP echo reply, id 6, seq 1, length 64
14:56:50.321104 00:0c:29:b1:cc:47 > aa:bb:cc:00:dc:00, ethertype IPv4 (0x0800), length 98: 192.168.99.100 > 192.168.99.2: ICMP echo request, id 6, seq 2, length 64
14:56:50.324450 aa:bb:cc:00:dc:00 > 00:0c:29:b1:cc:47, ethertype IPv4 (0x0800), length 98: 192.168.99.2 > 192.168.99.100: ICMP echo reply, id 6, seq 2, length 64Read top to bottom and the whole mechanism is visible: a broadcast to ff:ff:ff:ff:ff:ff asking who-has, a unicast is-at reply straight back, and only then does IP flow now that the destination MAC is known. If ping is failing and you never see the is-at, your problem is layer 2 and no amount of routing troubleshooting helps (the complete guide to what ping is really testing covers the ICMP side).
Layer 3: TTL, DF, identification. Those echo replies come back with ttl 255, which is IOS answering locally; a reply from further away arrives lower. The Don't Fragment bit plus a large packet is how you find MTU black holes (see using the DF bit to find the real path MTU). TTL 1 is normal for OSPF Hellos and suspicious on anything else.
Layer 4: flags, sequence, window. Where most diagnosis happens. A three-way handshake with the SSH banner exchange straight after:
$ sudo tcpdump -i ens224 -nn -c 6 'tcp and host 192.168.99.1 and port 22'
14:56:58.677331 IP 192.168.99.100.48124 > 192.168.99.1.22: Flags [S], seq 1351658335, win 64240, options [mss 1460,sackOK,TS val 3102991609 ecr 0,nop,wscale 7], length 0
14:56:58.681261 IP 192.168.99.1.22 > 192.168.99.100.48124: Flags [S.], seq 894975465, ack 1351658336, win 65535, options [mss 1460,sackOK,nop,nop,wscale 2,eol], length 0
14:56:58.681343 IP 192.168.99.100.48124 > 192.168.99.1.22: Flags [.], ack 1, win 502, length 0
14:56:58.682650 IP 192.168.99.100.48124 > 192.168.99.1.22: Flags [P.], seq 1:42, ack 1, win 502, length 41: SSH: SSH-2.0-OpenSSH_10.0p2 Debian-7+deb13u4
14:56:58.684111 IP 192.168.99.1.22 > 192.168.99.100.48124: Flags [P.], seq 1:20, ack 1, win 32768, length 19: SSH: SSH-2.0-Cisco-1.25
14:56:58.684164 IP 192.168.99.100.48124 > 192.168.99.1.22: Flags [.], ack 20, win 502, length 0Flags [S] then [S.] then [.] is the handshake (the dot is the ACK bit, so [S.] is SYN-ACK and [P.] is PSH-ACK). Note the sequence numbers: the SYN carries an absolute ISN and everything after it counts from 1, because tcpdump switches to relative numbering per direction once it has seen the opener. That is why a capture starting mid-session shows enormous sequence numbers and is far harder to read. Compare all of this with the [S] answered by [R.] earlier: same first packet, completely different verdict.
Payload, last. -A renders payload as ASCII and -X puts hex alongside. Reach for these only once the headers have told you the session works and the question has moved to what it carries.
Capture on Cisco IOS XE: Minimum Viable EPC
Embedded Packet Capture is four moving parts (a name, an attachment point, a filter and a buffer) and five lines gets you a usable capture:
! 1. define the capture and where it attaches
monitor capture CAP interface GigabitEthernet0/0/0 both
! 2. filter it - an inline match, or an existing ACL
monitor capture CAP match ipv4 host 192.168.99.100 host 192.168.99.2
! or: monitor capture CAP access-list SCAN-DETECT
! 3. bound it so it cannot run away
monitor capture CAP buffer size 10 circular
monitor capture CAP limit duration 60 packets 1000
! 4. run it, reproduce the problem, stop it
monitor capture CAP start
monitor capture CAP stopThen read it on the box or take it away:
show monitor capture CAP buffer brief
show monitor capture CAP buffer detailed
monitor capture CAP export flash:cap.pcap
no monitor capture CAPFour things catch people out. The filter is mandatory in practice even though the parser lets you skip it, because the buffer is device RAM and an unfiltered capture on a busy interface fills it in under a second. The both direction doubles what you collect, so use in or out once you know which way the interesting traffic goes. The buffer is usually circular, so a long run silently discards the beginning; bound it with limit duration rather than trusting yourself to stop it. And EPC punts copies of frames to a control-plane process, so it costs CPU in a way a hardware SPAN session does not. The export is a normal libpcap file, so the workflow ends where every other one does.
Filtering: BPF vs Display Filters
These are two languages doing two jobs, and conflating them wastes an enormous amount of time. Wireshark presents both in the same window, in different boxes, with no warning that the syntax is not interchangeable.
tcp port 80tcp.port == 80The consequence is a rule you should treat as absolute: capture wide, filter narrow on read. A BPF filter you got wrong cannot be undone, because the frames it rejected were never copied out of the kernel. A display filter you got wrong costs one keystroke. tcpdump gives you both, because BPF also works on read. One file, four questions, no re-capture:
$ tcpdump -nn -r res0017-lab-edge.pcap | wc -l
40
$ tcpdump -nn -r res0017-lab-edge.pcap 'icmp' | wc -l
12
$ tcpdump -nn -r res0017-lab-edge.pcap 'host 192.168.99.3' | wc -l
6
$ tcpdump -nn -r res0017-lab-edge.pcap 'port 80 or port 22' | wc -l
27Where BPF earns its keep is the byte-offset primitives, which match fields the named keywords do not cover. The most useful one in the language finds connection openers and nothing else:
$ sudo tcpdump -i ens224 -nn -vv -c 1 'tcp[tcpflags] & tcp-syn != 0 and tcp[tcpflags] & tcp-ack = 0'
14:57:13.608571 IP (tos 0xb8, ttl 64, id 59487, offset 0, flags [DF], proto TCP (6), length 60)
192.168.99.100.45308 > 192.168.99.2.22: Flags [S], cksum 0x47e6 (incorrect -> 0xcdf1), seq 2288597127, win 64240, options [mss 1460,sackOK,TS val 800626090 ecr 0,nop,wscale 7], length 0SYN set and ACK clear means genuine connection attempts with the SYN-ACK replies excluded, which is your answer to "who is initiating connections". It is also, incidentally, exactly the shape a port scan leaves behind:
$ tcpdump -nn -r res0030-synscan.pcap | head -12
16:03:10.968095 IP 192.168.99.101.47066 > 192.168.99.2.53: Flags [S], seq 3251516137, win 1024, options [mss 1460], length 0
16:03:10.968140 IP 192.168.99.101.47066 > 192.168.99.2.111: Flags [S], seq 3251516137, win 1024, options [mss 1460], length 0
16:03:10.968231 IP 192.168.99.101.47066 > 192.168.99.2.21: Flags [S], seq 3251516137, win 1024, options [mss 1460], length 0
16:03:10.968271 IP 192.168.99.101.47066 > 192.168.99.2.22: Flags [S], seq 3251516137, win 1024, options [mss 1460], length 0
16:03:10.968296 IP 192.168.99.101.47066 > 192.168.99.2.25: Flags [S], seq 3251516137, win 1024, options [mss 1460], length 0
16:03:10.968306 IP 192.168.99.101.47066 > 192.168.99.2.80: Flags [S], seq 3251516137, win 1024, options [mss 1460], length 0
16:03:10.972170 IP 192.168.99.2.53 > 192.168.99.101.47066: Flags [R.], seq 0, ack 3251516138, win 0, length 0
16:03:10.972248 IP 192.168.99.2.111 > 192.168.99.101.47066: Flags [R.], seq 0, ack 3251516138, win 0, length 0
16:03:10.972554 IP 192.168.99.2.21 > 192.168.99.101.47066: Flags [R.], seq 0, ack 3251516138, win 0, length 0
16:03:10.973236 IP 192.168.99.2.22 > 192.168.99.101.47066: Flags [S.], seq 3976364105, ack 3251516138, win 65535, options [mss 1460,sackOK,eol], length 0
16:03:10.973355 IP 192.168.99.101.47066 > 192.168.99.2.22: Flags [R], seq 3251516138, win 0, length 0
16:03:10.973630 IP 192.168.99.2.110 > 192.168.99.101.47066: Flags [R.], seq 0, ack 3251516138, win 0, length 0One source, one destination, many destination ports, all inside two milliseconds, each a lone SYN with no data. Closed ports answer RST; port 22 answers SYN-ACK and the scanner tears it down with a bare RST rather than completing the handshake, which is what makes it a half-open scan. The port scan types and what each one puts on the wire maps scanner flags onto exactly these signatures.
One honest caveat on clever filters. The popular recipe for finding traceroute probes, ip[8] < 64, is noisy on a router segment, because ip[8] is the TTL byte and IOS sends OSPF Hellos with TTL 1. Add the transport to fix it: udp and ip[8] < 64 and dst portrange 33434-33534, or icmp and ip[8] < 64 for the Windows flavour. Test byte-offset filters against real traffic before trusting them.
Security and Privacy: A pcap Is Sensitive Data
The moment you write a pcap you have created a file that may contain credentials, session tokens and personal data. Start with the demonstration, because nobody takes this seriously until they have seen it. Telnet was deliberately enabled on R3, a login performed, and the capture read back with -A:
$ tcpdump -nn -A -r telnet.pcap 'tcp port 23 and greater 60'
15:04:00.955095 IP 192.168.99.3.23 > 192.168.99.100.56194: Flags [P.], seq 13:55, ack 1, win 32768, length 42
E..R;?....7...c...cd.....$......P.......
User Access Verification
Username:
15:04:02.919204 IP 192.168.99.100.56194 > 192.168.99.3.23: Flags [P.], seq 45:51, ack 88, win 502, length 6
E...U.@.@..r..cd..c........F.$..P...G...admin
15:04:02.938662 IP 192.168.99.3.23 > 192.168.99.100.56194: Flags [P.], seq 93:105, ack 51, win 32755, length 12
E..4;P....7...c...cd.....$."...LP...o!..
Password:
15:04:04.924441 IP 192.168.99.100.56194 > 192.168.99.3.23: Flags [P.], seq 51:60, ack 105, win 502, length 9
E..1U.@.@..k..cd..c........L.$..P...G...Cisco123Username admin, password Cisco123, in plain view, no decryption and no specialist tooling. Telnet echoes every keystroke back from the server, so the credentials appear character by character in both directions. That is the entire argument for transport input ssh in one screenshot, and why killing telnet and managing over SSH is not a box-ticking exercise. The same exposure applies to every cleartext protocol still lurking in enterprise networks: HTTP, FTP, LDAP without TLS, syslog, and SNMPv1 and v2c, where the community string is a password sent in the clear on every poll (hence SNMPv3 with authentication and privacy).
Captures also record what you did not intend to collect. One on a user VLAN picks up browsing metadata and personal data, which drags you into GDPR, HIPAA, PCI DSS and your own acceptable use policy. The handling rules:
- Get authorisation in writing before capturing on production. "I was troubleshooting" is not a defence if you sniffed an HR subnet.
- Filter at capture time when the target is known. An expression scoped to the two hosts in the ticket collects the evidence and nothing else. This is the one legitimate reason to break the capture-wide rule.
- Use a small snaplen when you only need headers.
-s 96stores enough for L2 through L4 analysis and structurally cannot capture payload. - Treat the file as classified from the moment it exists. Encrypted at rest, access controlled, never attached to a ticket, never left in
/tmpon a jump host. - Set a retention window and honour it, and sanitise before sharing. Vendors and TAC rarely need payload, and an old pcap is pure liability.
One more thing a capture proves: source addresses are not identity. Scapy was used to send ICMP and TCP from four addresses that existed nowhere on the segment, and R2 logged them exactly as stamped:
R2# show logging | include 10.66.66.66|172.16.240.5|192.0.2.13|198.51.100.7
*Jul 24 23:11:41.914: %SEC-6-IPACCESSLOGP: list SCAN-DETECT permitted tcp 198.51.100.7(20) -> 192.168.99.2(22), 2 packets
*Jul 24 23:11:41.914: %SEC-6-IPACCESSLOGDP: list SCAN-DETECT permitted icmp 192.0.2.13 -> 192.168.99.2 (8/0), 3 packets
*Jul 24 23:11:41.914: %SEC-6-IPACCESSLOGDP: list SCAN-DETECT permitted icmp 10.66.66.66 -> 192.168.99.2 (8/0), 3 packets
*Jul 24 23:11:41.914: %SEC-6-IPACCESSLOGDP: list SCAN-DETECT permitted icmp 172.16.240.5 -> 192.168.99.2 (8/0), 3 packetsThe router cannot know the source is a lie, which is why unicast reverse path forwarding and anti-spoofing ACLs at the edge exist, and why an IP address in a log is a lead and not a conviction. The broader control set sits in the infrastructure security guide.
Common Failures That Ruin Captures
Dropped packets on a busy link. Check the drop counter before you analyse anything. A non-zero packets dropped by kernel means the ring buffer overflowed and your file has gaps you cannot see. The causes are self-inflicted: printing to a terminal instead of writing with -w, leaving name resolution on so every address triggers a DNS lookup (which also injects your own DNS queries into the capture), too broad a filter, or slow storage. Fixes in order of effect: always -nn, always -w file, tighten the filter, raise the buffer with -B, reduce the snaplen. On a saturated link, one host cannot capture it all and the tap point has to move to hardware.
Capturing on the wrong side of NAT. If the device translates, the flow has two identities and you have to know which one exists at your tap point. Filter for the private address on the outside interface and you see nothing at all, then spend twenty minutes convinced the traffic is being dropped. On a firewall, capture on both interfaces at once with the correct address on each and match the flows by timestamp and port. Which address to filter on is decided by how NAT actually rewrites a packet on IOS XE.
Snaplen truncation. The snapshot length is how many bytes of each frame get stored. Modern tcpdump defaults to 262144, effectively the whole frame, but older builds and plenty of embedded implementations default to 68 or 96 bytes. A short snaplen gives you headers and nothing else, and the analyser reports malformed packets because the payload it expects is missing. Use -s 0 when you need payload and a small value when you do not. The failure is silent, so make it a conscious decision.
Capturing your own SSH session. You SSH to the capture host, start tcpdump, and every packet you capture generates output sent back over SSH, which generates more packets to capture. Even without a runaway loop, your management session pollutes the file. The handshake shown earlier is a real example: that is the capture host's own SSH session. Exclude yourself:
$ tcpdump -nn -r res0017-lab-edge.pcap 'not proto ospf and not port 22'
14:57:56.051646 IP 192.168.99.100 > 192.168.99.1: ICMP echo request, id 8, seq 1, length 64
14:57:56.055154 IP 192.168.99.1 > 192.168.99.100: ICMP echo reply, id 8, seq 1, length 64
14:57:57.053534 IP 192.168.99.100 > 192.168.99.1: ICMP echo request, id 8, seq 2, length 64
14:57:57.056569 IP 192.168.99.1 > 192.168.99.100: ICMP echo reply, id 8, seq 2, length 64
14:57:58.064454 IP 192.168.99.100 > 192.168.99.3: ICMP echo request, id 9, seq 1, length 64
14:57:58.067156 IP 192.168.99.3 > 192.168.99.100: ICMP echo reply, id 9, seq 1, length 64not port 22 drops your management session, not proto ospf drops the routing chatter, and what remains is the traffic you were asked about. Build that exclusion in from the start.
The privilege-drop rotation trap. Rotating with -G (new file every N seconds) and -W (stop after N files) is the correct way to run an unattended capture, but on Debian it fails partway through even under sudo:
$ sudo tcpdump -i ens224 -nn -w 'rot-%Y%m%d-%H%M%S.pcap' -G 10 -W 2
listening on ens224, link-type EN10MB (Ethernet), snapshot length 262144 bytes
tcpdump: /home/j/plzcap/ev3/rot-20260724-150201.pcap: Permission denied
$ ls -l rot-*.pcap
-rw-r--r-- 1 tcpdump tcpdump 426 Jul 24 15:02 /home/j/plzcap/ev3/rot-20260724-150149.pcapLook at the owner of the file that did get written. tcpdump opens the capture socket as root and then drops to the unprivileged tcpdump user, which is Debian's default hardening. The first file is opened before the drop and succeeds; every rotation after it runs as a user with no write access to your directory. Keep the privileges, or write somewhere the drop-target user can:
$ sudo tcpdump -i ens224 -nn -w 'zrot-%Y%m%d-%H%M%S.pcap' -G 10 -W 2 -Z root
listening on ens224, link-type EN10MB (Ethernet), snapshot length 262144 bytes
Maximum file limit reached: 2
8 packets captured
8 packets received by filter
0 packets dropped by kernel
$ ls -l zrot-*.pcap
-rw-r--r-- 1 root root 650 Jul 24 15:02 /home/j/plzcap/ev3/zrot-20260724-150201.pcap
-rw-r--r-- 1 root root 560 Jul 24 15:02 /home/j/plzcap/ev3/zrot-20260724-150211.pcapTwo files ten seconds apart, and tcpdump exited on its own after the second. That self-termination is the point. -G on its own rotates forever and will eventually fill the disk on the box you are troubleshooting, which is a memorable way to turn one outage into two, so always pair it with -W and never run tcpdump unbounded over a remote session.
The Full Packet Analysis Cluster, in Reading Order
The first group teaches you to capture, the second gives you traffic worth capturing, and the third turns the skill around and points it at your own network.
Start here: capture on a Linux host
Capture on the network device
Generating traffic worth capturing
Reading attacks in your own logs
Frequently Asked Questions
What is the difference between a capture filter and a display filter?
A capture filter is BPF syntax (tcp port 80) compiled into the kernel and applied before packets are stored, so anything it rejects is gone forever. A display filter is Wireshark syntax (tcp.port == 80) applied to packets already in the file, so it is non-destructive. They are separate languages, even though Wireshark offers both in the same window. The rule that follows: capture wide, filter narrow on read.
Do I need root to run tcpdump?
You need elevated privileges of some kind, because opening a raw capture socket is privileged. Without them you get You don't have permission to perform this capture on that device, which is a permissions problem and not a filter problem. Run under sudo, or grant the binary cap_net_raw,cap_net_admin so named users can capture without full root. On Debian tcpdump then drops to an unprivileged user after opening the socket, which breaks rotating captures unless you add -Z root.
How do I capture packets on a Cisco router without an external sniffer?
Use Embedded Packet Capture. Attach a capture to an interface, give it a filter (an inline match statement or an existing ACL), bound it with a buffer size and duration limit, then monitor capture NAME start, reproduce the problem, and stop. Read it in place with show monitor capture NAME buffer brief, or export a real pcap with monitor capture NAME export flash:cap.pcap. Keep the filter tight: the buffer is device RAM and EPC costs control-plane CPU.
Why does my capture show fewer packets than the interface counters?
Three possibilities, and tcpdump's exit summary distinguishes them. If packets dropped by kernel is above zero, the capture process could not keep up. If received by filter is far below what the interface counted, your BPF filter is excluding traffic you meant to keep. If both numbers look right but the traffic is simply absent, you are on the wrong tap point and the frames never reached your NIC, which on a switched network is the default unless you arranged a mirror.
Is it safe to capture packets on a production network?
Capturing is passive and does not affect forwarding, so the risk is not to the traffic. It is to you and your data: a pcap can hold credentials from any cleartext protocol, session tokens, and personal data belonging to your users, which brings regulatory obligations with it. Get authorisation first, scope the filter to the hosts in the ticket, use a small snaplen when you only need headers, and delete the file when the incident closes. The other real risk is operational, because an unbounded rotating capture can fill the disk on the device you are troubleshooting.
How long should I capture for?
Long enough to include the event and a margin either side, and no longer. For a reproducible problem, start the capture, reproduce it, stop it, which is usually seconds. For something intermittent, run a bounded rotating capture with -G and -W so you keep a rolling window without filling the disk. Always start before you reproduce, because a capture beginning mid-session loses the handshake and with it the relative sequence numbering that makes the rest readable.
Key Takeaways
- The capture is the tiebreaker. A router reporting its HTTP server as Enabled while answering every SYN with a RST is a real result from this lab, and only the capture showed it.
- Tap point first, filter second. On a switched network you see your own traffic and broadcasts unless you arranged a mirror, an on-device capture point, or a position on the path. Promiscuous mode does not fix a bad tap point.
- Check the drop counter before you trust the file.
packets dropped by kernelabove zero means gaps you cannot see. Use-nn, write with-w, and keep the filter tight to hold it at zero. - BPF and display filters are different languages. BPF runs in the kernel before storage and is irreversible; display filters run afterwards and cost nothing to change. Capture wide, filter narrow on read.
- Bound every capture.
-cinteractively,-Gwith-Wunattended, and remember the privilege drop: rotation fails with Permission denied under plain sudo until you add-Z root. - A pcap is sensitive data from the moment it exists. One telnet login here yielded
adminandCisco123in plain view. Authorise, scope, encrypt and delete.
Start with getting fluent with tcpdump on a real host, then move the same skill onto the gear with on-device capture on IOS XE. Once you can read a capture, everything from what ping is really testing to the hardening controls that stop what you find stops being theory.