C9800 RADIUS Authentication Failures: Troubleshooting Guide
When wireless clients fail to authenticate on your Catalyst 9800 controller, the problem almost always points to RADIUS. Whether you're dealing with shared secret mismatches, server reachability issues, EAP timeouts, or policy misconfigurations, RADIUS authentication failures are among the most common issues you'll encounter in production wireless networks. This guide walks you through the systematic troubleshooting methodology and command set you need to isolate and resolve these problems quickly.
Understanding RADIUS Authentication on the C9800
The C9800 handles authentication through AAA (Authentication, Authorization, and Accounting), which you configure either locally or via an external RADIUS server. When a client associates with an access point, the AP sends the client's credentials to the controller, which then relays that authentication request to your RADIUS server (typically Cisco ISE, FreeRADIUS, or another AAA server). The RADIUS server validates the credentials and returns an access-accept or access-reject message. If anything breaks in this exchange—network connectivity, shared secrets, EAP negotiation, certificate validation—your clients cannot authenticate.
The critical components in this flow are (parenthetical notes on common failure points):
- Authenticator (the C9800) — relays client credentials to the RADIUS server; must have matching shared secrets configured
- Authentication Server (RADIUS) — validates credentials and returns policy attributes; must be reachable on the network
- EAP Negotiation — method-specific exchange between client and server (EAP-PEAP, EAP-TLS, etc.); certificate validation occurs here
- RADIUS Attributes & VSAs — policy decisions returned by server and interpreted by the controller
Common RADIUS Failure Scenarios
| Failure Scenario | Root Cause | Observable Symptom | Quick Check |
|---|---|---|---|
| Shared Secret Mismatch | C9800 and RADIUS server configured with different shared secrets | Access-Reject from server; Message-Authenticator validation fails | show run | include radius server; verify secret on ISE |
| Server Unreachable | RADIUS server IP not routable; firewall blocking UDP 1812/1813; server offline | No response from server; dead server detection triggers | Ping RADIUS server IP; check firewall ACLs for UDP 1812 |
| Dead Server Detection | Server exceeded failure threshold; too many timeouts or rejects | Server marked dead; clients still fail even if server recovers | show radius statistics; check deadtime and retry counts |
| EAP Timeout | Client not responding to EAP requests; network latency; certificate issues | EAP handshake hangs; client retries and eventually gives up | Capture on wireless link; check EAP message sequence timing |
| Certificate Validation Failure | Server cert expired, self-signed, or not trusted by client; CN mismatch | Client rejects during TLS tunnel setup (EAP-TLS, PEAP, EAP-FAST) | Verify ISE certificate chain; check client trusted CA store |
| VSA or Policy Mismatch | ISE returning attributes controller doesn't recognize; VLAN ID invalid | Auth succeeds but policy not applied; client gets wrong VLAN | Review RADIUS response; inspect cisco-av-pair attributes |
| AAA Method List Misconfiguration | WLAN security profile not linked to correct AAA authentication method | Controller doesn't send request to RADIUS; uses local auth instead | show wlan security; verify method list references exist |
| CoA Failure | ISE sending Change-of-Authorization but controller not listening or rejecting it | Policy changes on ISE don't reflect on controller; session not updated | Check CoA port 3799 reachability; verify CoA enabled on controller |
| Accounting Failures | Accounting requests dropped or not being sent; server offline | ISE shows no session accounting data; client billing/reporting gaps | show radius statistics accounting; verify accounting server config |
Step 1: Verify RADIUS Server Configuration and Reachability
Before diving into debug output, confirm basic network plumbing. Start by checking that your RADIUS server is configured and reachable.
C9800# show run | include radius
radius server ISE-Primary
address ipv4 10.1.1.100
key 7 MySharedSecret123
timeout 5
retransmit 3
radius server ISE-Backup
address ipv4 10.1.1.101
key 7 MySharedSecret123
timeout 5
retransmit 3
Verify that the shared secret is identical on both the controller and the RADIUS server. Even a single character difference will cause Message-Authenticator validation to fail. Next, test network reachability to your RADIUS server using the ping utility on the C9800:
C9800# ping 10.1.1.100
Sending 5, 100-byte ICMP Echos to 10.1.1.100, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), roundtrip min/avg/max = 1/2/5 ms
If ping fails, check your routing table, default gateway, and wired network connectivity. Use traceroute to verify the path to the RADIUS server if the network is complex. Additionally, confirm that your firewall or ACLs are not blocking UDP port 1812 (RADIUS authentication) or 1813 (RADIUS accounting).
Step 2: Check RADIUS Server Statistics and Dead Server Status
The C9800 maintains state for each RADIUS server, including whether it is alive or dead. If a server exceeds the configured retry and timeout limits, the controller marks it as dead and stops sending requests to it temporarily. Use this command to review server status:
C9800# show radius statistics
Server Address State Timeouts Retransmits Requests Accepts Rejects
10.1.1.100 ALIVE 0 0 145 123 22
10.1.1.101 DEAD 15 45 50 0 0
If a server is marked DEAD, review the failure count and then run:
C9800# clear radius statistics
Then re-test client authentication. If the server recovers and starts responding, the ALIVE state will be restored automatically. However, if a RADIUS server is legitimately down, configure a failover server or bring the primary server back online. The dead server detection deadtime is configurable (default 0 means immediate failover):
C9800(config)# radius-server deadtime 30
This deadtime value (in minutes) specifies how long the controller waits before retrying a dead server. A value of 0 means the server is tried again with the next request.
Step 3: Enable and Interpret Radioactive Traces for Auth Flow Debugging
When basic checks pass and the server is reachable, enable the C9800's radioactive trace to capture the full authentication flow. Radioactive tracing provides granular visibility into EAP message sequences, RADIUS packet exchanges, and AAA decisions.
C9800# debug wireless aaa events
C9800# debug wireless aaa details
C9800# debug eap all
Next, attempt authentication from a wireless client. The debug output will show you:
- EAP-Start and EAP-Request messages
- EAP method negotiation (e.g., PEAP, TLS, MSCHAPv2)
- RADIUS Access-Request packets sent to the server
- RADIUS Access-Accept or Access-Reject responses
- AAA policy application to the client session
Example trace output (paraphrased for clarity):
*Jan 29 21:51:31.736: %CAPWAPAAA_EMERG_TRACE_MESSAGE-5-AP_JOIN_DISJOIN:
Client MAC: 00:11:22:33:44:55 IP: 10.5.1.1[5246] Event: Association
*Jan 29 21:51:32.102: %CAPWAPAAA_EMERG_TRACE_MESSAGE-5-AP_JOIN_DISJOIN:
EAP Start Received; starting auth negotiation
*Jan 29 21:51:32.456: [radius] (25734): (info): RADIUS: Send Access-Request to
10.1.1.100:1812 id 59, len 384
*Jan 29 21:51:32.498: [radius] (25734): (info): RADIUS: Received from id
1812/59 192.168.1.99:0, Access-Accept, len 185
*Jan 29 21:51:32.654: %CAPWAPAAA_EMERG_TRACE_MESSAGE-5-AP_JOIN_DISJOIN:
EAP Success Sent; client authenticated
Look for these key indicators in the trace:
- RADIUS: Send Access-Request — controller is sending requests; if absent, check AAA method list binding
- RADIUS: Access-Accept / Access-Reject — server response; Reject means credentials or policy failed on server
- Message-Authenticator validation failed — shared secret mismatch (this message appears in logs even if trace is off)
- EAP timeout — client or server not responding within configured timeout; usually EAP-TLS cert issue
- EAP Failure Sent — authentication failed; correlate with preceding messages to identify the failure point
Step 4: Capture and Analyze RADIUS Packets
For deeper investigation, especially when dealing with shared secret or attribute issues, capture the RADIUS packets themselves. On the C9800, you can use the packet capture feature in the Troubleshooting dashboard or enable a capture on the controller's management port (if your network setup permits).
Via CLI, you can export platform captures:
C9800# show platform hardware chassis active qfp packet-trace packet all decode | redirect bootflash:c9800-radius-capture.txt
Alternatively, capture traffic on the RADIUS server or a SPAN port if the C9800 is connected to a switch. Use Wireshark or tcpdump to inspect RADIUS packets. Look for:
- Access-Request (code 1) — outbound from controller to server; contains User-Name, User-Password (encrypted), NAS-IP-Address, etc.
- Access-Accept (code 2) — inbound from server; contains policy attributes (VLAN, ACL, etc.)
- Access-Reject (code 3) — inbound from server; authentication denied; check Reply-Message attribute for reason
- Message-Authenticator attribute — HMAC-MD5 of entire packet using shared secret; if controller and server secrets differ, validation fails
In Wireshark, filter for RADIUS packets:
radius.code == 1 or radius.code == 2 or radius.code == 3
Expand the packet tree and inspect the attributes. If you see Access-Reject, check the Reply-Message attribute for a server-provided error (e.g., "Invalid Credentials" or "Certificate Invalid").
Step 5: Diagnose Shared Secret Mismatches
A shared secret mismatch is one of the most common (and frustrating) RADIUS issues. The shared secret is used to encrypt the User-Password attribute and to compute the Message-Authenticator HMAC. If the secrets don't match:
- The controller sends Access-Request with encrypted password; server decrypts with its secret, gets garbage
- Server computes Message-Authenticator with its secret; controller verifies with its secret, validation fails
- Server rejects the request or controller drops the response
To verify the shared secret, display the running config (carefully—the key is encrypted on display):
C9800# show run | include radius
radius server ISE-Primary
address ipv4 10.1.1.100
key 7 MySharedSecret123
The key is shown in type-7 encrypted form. If you need to verify the actual secret, compare it against the RADIUS server configuration. On Cisco ISE:
- Navigate to Administration > Network Resources > Network Devices
- Find the C9800 entry
- Check the Shared Secret field
If the secrets do not match, update the controller:
C9800(config)# radius server ISE-Primary
C9800(config-radius-server)# key MyNewSharedSecret123
C9800(config-radius-server)# exit
Then test authentication again. Clear any stale debug output first:
C9800# undebug all
C9800# clear log
Step 6: Diagnose EAP Timeout and Certificate Issues
EAP-based authentication (PEAP, EAP-TLS, EAP-FAST) requires a secure tunnel between the client and the RADIUS server. If the client's certificate validation fails or the server's certificate is not trusted, the client will hang on the TLS handshake.
Common certificate issues include:
- Server certificate expired — client checks validity period and rejects
- Server certificate not signed by a trusted CA — client's trusted CA store doesn't include the issuing CA
- Common Name (CN) mismatch — client expects server FQDN to match certificate CN; hostname verification fails
- Client certificate invalid or revoked — EAP-TLS requires client cert; ISE policy may reject revoked certs
To inspect ISE certificates on the C9800, request a CSR and view the server cert chain. Verify the cert is valid and that the client has the root CA in its trust store. For EAP-TLS, ensure the client and server certificates are both valid and that the client certificate is enrolled to the user/device.
Monitor EAP timeouts using debug:
C9800# debug wireless aaa events
C9800# debug eap all
If you see "EAP timeout" in the trace, it usually means the client did not respond to the server's TLS request. This happens when:
- The client is attempting to validate the server cert and fails (most common)
- The RADIUS server is slow or unresponsive
- The wireless link is lossy and EAP frames are being dropped
Capture the wireless frames between the client and AP to verify EAP message sequence. Use a packet analyzer with 802.11 frame decoding to see if the client is sending EAP-Response messages. If the client is not responding, the issue is likely on the client side (certificate validation or EAP method not supported).
Step 7: Verify AAA Method List Configuration
If RADIUS server is reachable and credentials are correct, but clients still fail, check that the WLAN security profile is correctly linked to the AAA authentication method. The method list tells the controller which servers to use and in what order.
C9800# show wlan security
Name : Corporate
Auth Method Name : corporate-eap
AAA Override : false
C9800# show run | include aaa authentication
aaa authentication network-access corporate-eap local
aaa authentication network-access corporate-eap group radius
aaa authorization network-access corporate-eap local
aaa authorization network-access corporate-eap group radius
Verify that the method list references a valid AAA server group:
C9800# show run | include aaa group
aaa group server radius ISE-Servers
server name ISE-Primary
server name ISE-Backup
If the method list is missing or misconfigured, add or update it:
C9800(config)# aaa group server radius ISE-Servers
C9800(config-radius)# server name ISE-Primary
C9800(config-radius)# server name ISE-Backup
C9800(config-radius)# exit
C9800(config)# aaa authentication network-access corporate-eap group radius
Step 8: Check for Policy Mismatch and RADIUS Attributes
Once authentication succeeds, the RADIUS server returns attributes that define the client's network policy (VLAN, ACL, session timeout, etc.). If the server returns attributes the C9800 doesn't recognize or interprets incorrectly, clients may get the wrong VLAN or access rights.
In debug output, look for lines like:
*Jan 29 21:51:32.498: [radius] (25734): (info): RADIUS: Cisco AVpair [1]
25 "service-type=Call Check"
*Jan 29 21:51:32.498: [radius] (25734): (info): RADIUS: Cisco AVpair [26]
30 "psk-mode=ascii"
The RADIUS server can return Cisco-proprietary attributes (cisco-av-pair) or standard IETF attributes like Tunnel-Private-Group-ID (VLAN). If you see unexpected attributes or the policy is not applied, check the ISE authorization policy. For MAC address filtering (iPSK), ISE must return:
cisco-av-pair = "psk-mode=ascii"User-Password = "<PSK>"(encrypted in RADIUS packet)
If ISE is not returning these, the policy profile is not correctly configured on the server.
Step 9: Diagnose CoA (Change of Authorization) Failures
ISE can push policy changes to the controller via CoA (Change of Authorization) messages on UDP port 3499. If CoA fails, policy updates won't reach the client until the session is re-authenticated.
Verify CoA is enabled:
C9800# show run | include coa
radius-server coa port 3499
radius-server coa key MyCoASecret
Test CoA reachability from ISE. On ISE, navigate to Administration > Network Resources > Network Devices and verify that the C9800 entry has "CoA Port" set to 3499 and a shared secret configured. The C9800 shared secret and ISE shared secret must match for CoA messages to be processed.
Monitor incoming CoA with debug:
C9800# debug wireless aaa coa all
Attempt a policy change on ISE (e.g., change a user's VLAN in a profiling rule). Check the debug output for incoming CoA-Request messages. If CoA fails, check the controller's syslog for error messages (e.g., "Invalid Request Authenticator" = shared secret mismatch).
Step 10: Validate Accounting Configuration
Accounting ensures that ISE has visibility into who is connected and how long they've been on the network. If accounting requests are not reaching ISE, session records will be incomplete and billing/reporting will fail.
C9800# show run | include radius accounting
radius server ISE-Primary
accounting port 1813
C9800# show radius statistics accounting
Server Address Requests Responses Retransmits Pending
10.1.1.100 245 245 0 0
If Pending is non-zero, accounting requests are being dropped or the server is not responding. Check that the accounting port (1813) is reachable and that ISE's Accounting configuration includes the C9800's management IP.
Real-World Troubleshooting Example: Step-by-Step Walkthrough
A user reports that clients can associate but cannot authenticate. The AP shows "802.1X Authentication Failed" in the syslog. Here's the systematic approach:
- Ping the RADIUS server:
C9800# ping 10.1.1.100→ Success. Network is OK. - Check RADIUS server status:
C9800# show radius statistics→ ISE-Primary shows ALIVE, low failure count. Server is responsive. - Enable debug and attempt auth:
C9800# debug wireless aaa events→ Trace shows "RADIUS: Send Access-Request" but then "Access-Reject" response. - Capture RADIUS response: The Reply-Message attribute says "Certificate Not Trusted". Issue is on client side, not controller.
- Check client certificate trust store: Client (Windows device) does not have ISE's root CA in the trusted root store. Add ISE root CA to client, retry. Authentication succeeds.
Relevant Show Commands and Output Summary
| Command | Purpose | Key Output Fields |
|---|---|---|
show radius statistics |
RADIUS server state and request/response counts | State (ALIVE/DEAD), Timeouts, Retransmits, Accepts, Rejects |
show run | include radius |
Display RADIUS server configuration | address, key (encrypted), timeout, retransmit |
show wlan security |
WLAN AAA method list binding | Auth Method Name, AAA Override, AKM |
show run | include aaa authentication |
AAA method list definition | network-access method_name group radius |
show aaa authentication method |
Detailed AAA method configuration | Server group, fallback, timeout |
debug wireless aaa events |
High-level AAA event trace | Association, EAP Start, Auth Success/Failure |
debug wireless aaa details |
Detailed AAA message-level trace | RADIUS Access-Request/Accept/Reject, attributes |
debug eap all |
EAP frame and method negotiation | EAP-Request/Response, method type, TLS state |
ping <RADIUS-IP> |
Test network reachability to RADIUS server | Success rate, round-trip times |
Key Takeaways
- RADIUS server reachability is foundational. If you cannot ping the server or UDP 1812 is blocked, nothing else matters. Verify routing and firewall rules first.
- Shared secrets must match exactly. A single character difference will cause silent authentication failures and difficult-to-diagnose issues. Always double-check against the server configuration.
- Dead server detection is a feature, not a bug. If a server is marked DEAD, confirm it is actually running and responding. Check the Requests/Rejects count in show radius statistics to understand why it was marked dead.
- EAP timeouts usually mean certificate issues on the client. Enable debug eap all and look at the client-side wireless trace to see if the client is responding to server requests. If not, the client is rejecting the server certificate.
- Method list misconfiguration is surprisingly common. Always verify that your WLAN security profile references an existing AAA method list and that the method list points to a valid RADIUS server group.
- Use radioactive traces to follow the authentication flow. The trace output shows exactly where authentication fails (before RADIUS, at RADIUS, or during policy application). This is far more reliable than guessing from syslog entries.
- RADIUS attributes control authorization policy. If authentication succeeds but clients get the wrong VLAN or ACL, the server is returning unexpected attributes. Verify the ISE authorization policy matches your intent.
- CoA failures are silent by default. Enable debug wireless aaa coa all to see if ISE's policy change requests are reaching the controller. If not, check UDP 3499 reachability and shared secret alignment.
- Accounting requests validate end-to-end connectivity. If Accounting shows high Pending or Retransmits, the server is not acknowledging. Confirm the accounting port and server IP are correct and reachable.
- Packet captures are your friend when debug output is not enough. Wireshark or tcpdump on the RADIUS server can reveal message-authenticator mismatches, attribute encoding issues, or server-side policy rejections that debug traces won't show in real time.