BGP Session Authentication: MD5 and TCP-AO

An unauthenticated BGP session is vulnerable to TCP-based attacks: a spoofed RST can tear down a peering session, and a spoofed packet on port 179 could potentially inject false routes. BGP authentication adds a cryptographic signature to every TCP segment, ensuring that only packets from the legitimate peer are accepted. MD5 (RFC 2385) has been the standard for decades, but it has known weaknesses. TCP-AO (RFC 5925) is the modern replacement — stronger algorithms, key rollover support, and protection against replay attacks.

MD5 Authentication (RFC 2385)

MD5 authentication adds a 16-byte MD5 digest to every TCP segment. Both peers must share the same password, and any segment with an invalid digest is silently dropped.

Configuration

! R1-HQ
router bgp 65001
 neighbor 172.16.0.2 password 0 MySecretP@ss!

! ISP-A-PE1
router bgp 65010
 neighbor 172.16.0.1 password 0 MySecretP@ss!

Both sides must have identical passwords. If you add or change the password on an existing session, the session will flap — the TCP connection tears down (existing segments fail auth) and re-establishes with the new key.

Encrypted Password Display

By default, show running-config displays the password in Type 7 encryption (easily reversible). Use service password-encryption to at least obscure it, but treat Type 7 as security-through-obscurity only.

R1-HQ# show running-config | include password
 neighbor 172.16.0.2 password 7 05080F1C2243

MD5 Limitations

• Weak algorithm: MD5 has known collision vulnerabilities. While practical attacks against BGP MD5 auth in transit are unlikely, it doesn't meet modern cryptographic standards.
• No key rollover: Changing the password requires coordinated changes on both sides simultaneously, causing a session flap. In practice, many operators set the password once and never rotate it.
• No replay protection: MD5 auth doesn't protect against TCP segment replay attacks (though the TCP sequence number check provides some implicit protection).

TCP Authentication Option (TCP-AO, RFC 5925)

TCP-AO is the modern replacement for MD5. It supports multiple algorithms (HMAC-SHA-1-96, AES-128-CMAC-96), key rollover without session disruption, and protects against replay attacks. IOS XE supports TCP-AO on recent platforms (17.x on ASR/ISR/Catalyst 8000).

Configuration with Key Chains

! Define the key chain
key chain BGP-KEYS
 key 1
  key-string SecureKey2025!
  send-lifetime 00:00:00 Jan 1 2025 00:00:00 Jul 1 2025
  accept-lifetime 00:00:00 Jan 1 2025 00:00:00 Aug 1 2025
  cryptographic-algorithm hmac-sha-1-96
 key 2
  key-string NewSecureKey2025!
  send-lifetime 00:00:00 Jun 1 2025 infinite
  accept-lifetime 00:00:00 Jun 1 2025 infinite
  cryptographic-algorithm hmac-sha-1-96
!
! Apply to TCP-AO profile
tcp ao keychain BGP-KEYS profile BGP-AO-PROFILE
!
! Apply to BGP neighbor
router bgp 65001
 neighbor 172.16.0.2 ao BGP-AO-PROFILE

Key features of this configuration:

• Key rollover: Key 1 and Key 2 have overlapping lifetimes. During June 2025, both keys are valid for acceptance but only Key 2 is used for sending. This allows hitless key rotation — change the sending key on one side, then the other, without session disruption.
• Accept lifetime wider than send: The accept window is intentionally longer than the send window to handle clock skew between peers and give operators time to complete the rollover on both sides.
• Algorithm selection: HMAC-SHA-1-96 is the most widely supported. AES-128-CMAC-96 is stronger but less commonly implemented.

GTSM (Generalized TTL Security Mechanism)

While not authentication per se, GTSM (RFC 5082) provides an additional layer of session protection by enforcing that BGP packets must have TTL 255 (for directly connected peers). This makes remote spoofing much harder — a spoofed packet from a remote attacker would have a lower TTL after traversing the network:

router bgp 65001
 neighbor 172.16.0.2 ttl-security hops 1

This replaces ebgp-multihop — you cannot use both simultaneously. GTSM checks that incoming packets have TTL >= 254 (255 minus the configured hop count). Combined with MD5 or TCP-AO, GTSM provides defense in depth.

Verification

R1-HQ# show ip bgp neighbors 172.16.0.2 | include auth
  TCP MD5 authentication is enabled
  TCP authentication AO is enabled (profile: BGP-AO-PROFILE)

R1-HQ# show ip bgp neighbors 172.16.0.2 | include TTL
  External BGP neighbor may be up to 1 hop away.
  TTL security hops: 1

R1-HQ# show key chain BGP-KEYS
Key-chain BGP-KEYS:
    key 1 -- text "..."
        accept lifetime (00:00:00 Jan 1 2025) - (00:00:00 Aug 1 2025)
        send lifetime (00:00:00 Jan 1 2025) - (00:00:00 Jul 1 2025)
    key 2 -- text "..."
        accept lifetime (00:00:00 Jun 1 2025) - (infinite)
        send lifetime (00:00:00 Jun 1 2025) - (infinite)

Best Practices

• Use MD5 authentication on ALL eBGP sessions as a minimum baseline.
• Migrate to TCP-AO when both sides support it — key rollover alone justifies the migration.
• Enable GTSM (ttl-security) on directly connected eBGP sessions for defense in depth.
• For iBGP sessions, authentication is less critical (traffic stays within your network) but still recommended if the iBGP path crosses untrusted segments.
• Document passwords in a secure password manager. Coordinate password changes with your peering partners during maintenance windows.

Troubleshooting

Key Takeaways

• MD5 authentication is the baseline for BGP session security — enable it on all eBGP sessions.
• TCP-AO is the modern replacement with stronger algorithms, key rollover, and replay protection. Migrate when supported.
• GTSM (ttl-security) prevents remote spoofing by requiring TTL 255 on incoming BGP packets. Use alongside authentication.
• Key rollover is the main operational advantage of TCP-AO over MD5 — no session disruption during password rotation.
• Always coordinate authentication changes with your peering partner to avoid extended outages.