Lab ts-nf-01 - Troubleshooting Connectivity Tickets (L1/L2/L3)

The build labs in the PingLabz CCNA Labs library teach you to configure a network from a blank slate. This series does the opposite. You inherit a network that someone else already broke, you get a ticket that describes a symptom and nothing else, and your job is to localize the fault and fix it. That is the skill the CCNA exam tests hardest and the one most study guides skip: not "how do I configure OSPF," but "the network is down, where do I even start."

This is the free preview of the PingLabz CCNA Troubleshooting Labs. It runs on the same five-node PingLabz CCNA Base Topology as the rest of the library, so it fits inside Cisco Modeling Labs Free. Every command output on this page was captured from that topology running on Cisco IOS XE, broken on purpose and then fixed.

How these troubleshooting labs work

Each lab is a small ticket queue. You import the topology, start it, and work the tickets in order. A ticket gives you a symptom in the user's words and a single success criterion. It does not tell you the cause. You reproduce the symptom, climb the OSI stack until you find the layer that is lying to you, fix it, and verify. After each ticket we show the root cause and the exact fix so you can check your work.

The topology

Three routers (R1, R2, R3), one managed switch (SW1), one host (HOST1), and an unmanaged switch (SW2). R1 and R2 share the LAN 10.20.0.0/24 through SW1. R2 reaches R3 over the point-to-point link 10.30.30.0/30. Each router has a loopback: 10.255.0.1, 10.255.0.2, 10.255.0.3. In the healthy state, static routing ties it together and R1 can reach every loopback. You log in as pinglabz / PingLabz!23.

Lab setup: this topology boots with all three faults already in place. On a small network faults mask each other, so work bottom-up: fix the most local, most fundamental fault first, then re-test before moving on. That order matters here. All your testing starts at R1, and R1 itself carries the two most upstream faults, so until you fix them R1 cannot even see the third one. The tickets below are sequenced the way the network forces you to solve them.

What you will learn

• The bottom-up method: when end-to-end ping fails, climb the stack from Layer 1 instead of guessing, and fix the most local fault before chasing a remote one.
• How to read the difference between a timeout (.) and an unreachable (U) in ping output, and why that one character tells you whether to look local or downstream.
• The four commands that expose each layer: show ip interface brief, show ip route, show ip arp, and traceroute.
• Why an interface in up/up can still drop traffic, and how a subnet mask becomes a connectivity fault that hides every other problem behind it.

Ticket 1: "A new tech renumbered R1 and now nothing works"

Reported symptom: "We changed R1's LAN address during a renumber. Now R1 can't reach anyone on the LAN."
Success criterion: R1 can ping its neighbor 10.20.0.2.

Start at R1, because that is where every other test will originate too. Confirm the symptom and source from the LAN address the tech configured:

R1# ping 10.20.0.2 source 10.20.0.129
.....
Success rate is 0 percent (0/5)

Dots, not U. A timeout, not an active rejection: the packet is dying silently and nobody is sending anything back. Climb from the bottom. Is the interface up?

R1# show ip interface brief | include Ethernet0/0
Ethernet0/0   10.20.0.129   YES TFTP   up                    up

Up and up. So this is not Layer 1 or Layer 2. The interface is healthy and forwarding. But R1 has no idea how to reach its own neighbor:

R1# show ip route 10.20.0.2
% Subnet not in table

How can a directly connected neighbor not be in the table? Look at the interface in detail:

R1# show running-config interface Ethernet0/0
interface Ethernet0/0
 description LAN to SW1 (10.20.0.0/24)
 ip address 10.20.0.129 255.255.255.128

The description says 10.20.0.0/24. The configured mask is 255.255.255.128, a /25. With a /25, R1 believes its connected subnet is 10.20.0.128/25 (hosts .129 to .254). The neighbor at 10.20.0.2 falls in the other half, 10.20.0.0/25, so R1 thinks it is on a different network entirely and never even tries to ARP for it. Worse, this one fault hides the other two: every static route R1 has points at a next-hop (10.20.0.2 or 10.20.0.99) that now looks off-subnet, so none of those routes are even installed. Fix this first or nothing else you test from R1 will make sense.

Root cause: wrong subnet mask. The interface description (the intended /24) contradicts the configured /25.
Fix:

R1(config)# interface Ethernet0/0
R1(config-if)# ip address 10.20.0.1 255.255.255.0

Re-test the neighbor:

R1# ping 10.20.0.2 source 10.20.0.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/3/5 ms

This is the trap of an up/up interface. Layer 1 and Layer 2 are fine, so the instinct to check cables and line protocol leads nowhere. The mask is a Layer 3 fault hiding behind a perfectly healthy physical link, and on this lab it is the fault that has to go first.

Ticket 2: "R1 still can't reach the R3 site"

Reported symptom: "The LAN works again, but R1 still cannot reach anything at the R3 site."
Success criterion: R1 can ping 10.255.0.3.

Now that R1 can talk to the LAN, test the far loopback:

R1# ping 10.255.0.3 source 10.255.0.1
.....
Success rate is 0 percent (0/5)

Dots again, so the packet is dying locally on R1, not being rejected downstream. Look at the route R1 is using:

R1# show ip route 10.255.0.3
Routing entry for 10.255.0.3/32
  Known via "static", distance 1, metric 0
  Routing Descriptor Blocks:
  * 10.20.0.99
      Route metric is 0, traffic share count is 1

R1 is sending traffic for 10.255.0.3 to a next-hop of 10.20.0.99. There is no such device on the LAN. Prove it:

R1# show ip arp 10.20.0.99
Protocol  Address          Age (min)  Hardware Addr   Type   Interface
Internet  10.20.0.99              0   Incomplete      ARPA

Incomplete means R1 ARPed for 10.20.0.99 and nobody answered. The next-hop is a ghost. The correct gateway to R3 is R2 at 10.20.0.2.

Root cause: the static route points at a non-existent next-hop (10.20.0.99 instead of 10.20.0.2).
Fix:

R1(config)# no ip route 10.255.0.3 255.255.255.255 10.20.0.99
R1(config)# ip route 10.255.0.3 255.255.255.255 10.20.0.2

A route is only as good as its next-hop. If ARP for the next-hop comes back Incomplete, the route is pointing at nothing, and the symptom is a silent timeout because R1 never gets a frame onto the wire.

Ticket 3: "I get a different failure to R3 now"

Reported symptom: "R1 used to time out reaching the R3 site. Now it fails faster and differently."
Success criterion: R1 can ping 10.255.0.3.

Same target, different fingerprint. Look at the ping carefully:

R1# ping 10.255.0.3 source 10.255.0.1
U.U.U
Success rate is 0 percent (0/5)

This time it is U, not dots. A router in the path actively sent back an ICMP unreachable. The packet is getting somewhere and being turned away, not vanishing into a black hole. That is a downstream problem now, not a local one. Find out how far it gets:

R1# traceroute 10.255.0.3 source 10.255.0.1
Tracing the route to 10.255.0.3
VRF info: (vrf in name/id, vrf out name/id)
  1 10.20.0.2 4 msec 3 msec 3 msec
  2  *  !H  *

It reaches R2 (10.20.0.2) cleanly at hop 1, then R2 itself returns !H (host unreachable). The fault is at or beyond R2. Move to R2 and check the interfaces:

R2# show ip interface brief | exclude unassigned
Interface              IP-Address      OK? Method Status                Protocol
Ethernet0/0   10.20.0.2     YES TFTP   up                    up
Ethernet0/1   10.30.30.1    YES TFTP   administratively down down
Loopback0     10.255.0.2    YES TFTP   up                    up

There it is. Ethernet0/1, the link to R3, is administratively down. Someone shut it. The route that depends on it has nowhere to go:

R2# show ip route 10.255.0.3
% Subnet not in table

Root cause: R2 Ethernet0/1 was administratively shut down.
Fix:

R2(config)# interface Ethernet0/1
R2(config-if)# no shutdown

Re-test from R1 and you are finally back to !!!!!:

R1# ping 10.255.0.3 source 10.255.0.1
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 3/4/6 ms

The lesson worth keeping: ..... sends you looking at the local forwarding decision and ARP on the router you are testing from; U.U.U sends you looking downstream for an interface or routing problem on another device. One character of ping output tells you which router to log into next. On this lab the two local faults on R1 came first by necessity, and only once they were gone could the downstream U even appear.

Troubleshooting matrix

Key takeaways

• When end-to-end ping fails, do not guess. Start at the bottom of the stack and fix the most local, most upstream fault first; on a small network it is probably masking the others.
• Read ping output as evidence. . versus U tells you whether to look local or downstream before you log into anything.
• up/up rules out Layers 1 and 2. It does not rule out a Layer 3 addressing mistake, and a wrong mask can hide every route on the box.
• A route is only as good as its next-hop. If ARP for the next-hop is Incomplete, the route is pointing at nothing.

This is the free preview of the troubleshooting series. The rest of the queue, by exam pillar, lives in the PingLabz CCNA Labs library: Layer 2 and VLAN tickets, OSPF tickets, DHCP and NAT tickets, security tickets, and a multi-fault capstone outage. If you want the general method behind all of them, the build-lab nf-11 Troubleshooting Layer Symptoms walks through the seven-step escalation drill.