pyATS and Genie: Diff Cisco Operational State, Not Config

You pushed a change at 02:00. The config looks right in show run. So why is half the branch black? Because config text is not state. The line went in and the interface never came up, or an adjacency reset and never came back, and nothing you diffed would have told you. Config diffing answers "did my text land". It does not answer "is this device still doing what it was doing an hour ago".

That second question is what Cisco pyATS and Genie were built for, and it is the part of a Python-based network automation toolchain most people skip past on their way to writing yet another config pusher. Genie snapshots a device's live operational state, lets you make a change, snapshots again, and hands you a structured diff of what actually moved.

Everything below was captured on real gear: a Debian 13 VM running pyATS 26.6, Genie 26.6 and Unicon 26.6 on Python 3.13.5, driving three iol-xe routers and an ioll2-xe switch on IOS-XE 17.18.2 in Cisco Modeling Labs.

Installing pyATS in 2026: the old warning is dead

Nearly every pyATS tutorial older than about a year opens with a warning about the install. Pin your Python version, expect C extensions to compile, do not even try the newest interpreter. Ignore all of it, because it is out of date. pip install "pyats[full]" now pulls cp313 wheels, and on Python 3.13.5 the whole stack (Genie, Unicon, the parser libraries, the traffic-generator bindings) installed with no compiler present and no errors. The only prerequisite was the venv module:

sudo apt-get install -y python3.13-venv
python3 -m venv ~/venvs/plz
~/venvs/plz/bin/pip install "pyats[full]"
~/venvs/plz/bin/pyats version check

That reports pyats 26.6, genie 26.6 and unicon 26.6. If you put pyATS off because you remembered the install being a fight, the fight is over.

The SSH problem that stops everyone before line one

Here is the failure you will actually hit, and it has nothing to do with pyATS. A modern OpenSSH client will not talk to a 2020-era Cisco image. It dropped SHA-1 host keys and the older key exchange groups years ago, and IOS is still offering exactly those. Your first .connect() dies with unicon.core.errors.EOF or Host key verification failed, and because the traceback comes out of Unicon, most people conclude pyATS is broken and go do something else.

It is not broken. Your SSH client is refusing to negotiate. Fix it once in ~/.ssh/config:

Host 192.168.99.*
    StrictHostKeyChecking no
    UserKnownHostsFile /dev/null
    KexAlgorithms +diffie-hellman-group14-sha1
    HostKeyAlgorithms +ssh-rsa
    PubkeyAcceptedAlgorithms +ssh-rsa
    PubkeyAuthentication no

Three things worth understanding rather than pasting blindly. The + prefix appends to the client's default list instead of replacing it, so you are widening what you accept, not downgrading everything you connect to. Scoping the stanza to your lab subnet (not Host *) keeps that widening off production and internet SSH. And PubkeyAuthentication no matters because if your agent has keys loaded, the client burns its authentication attempts offering them and IOS drops the session before you reach the password prompt.

The same stanza is what makes netmiko and NAPALM work from that box, so if you are building a netmiko SSH automation script alongside this, you only do it once.

The testbed file: your inventory, in YAML

pyATS does not take a list of IPs. It takes a testbed: a YAML file describing devices, what OS they run, and how to reach them. The os: key is the important one, because it tells Unicon which plugin to load and therefore how to drive the CLI prompt state machine. You never write expect logic.

testbed:
  name: PingLabz-Research-VM-Automation-Edge
  credentials:
    default:
      username: admin
      password: Cisco123

devices:
  R1:
    os: iosxe
    type: router
    platform: iol
    connections:
      cli:
        protocol: ssh
        ip: 192.168.99.1

Sanity-check it with pyats validate testbed testbed.yaml before you write any Python. Loading and connecting is then three lines, and learn_hostname=True makes Unicon read the real hostname off the prompt instead of trusting your YAML label, which is how you catch drifted inventory:

from genie.testbed import load
tb = load("testbed.yaml")
r1 = tb.devices["R1"]
r1.connect(log_stdout=False, learn_hostname=True)

Each connected device then carries its resolved attributes:

DEV    os         platform  type      connected hostname learned
R1     iosxe      iol       router    True      R1
SW1    iosxe      iol       switch    True      SW1

parse(): show output becomes JSON you can index

Two methods do the reading. execute() returns raw text exactly as the device printed it. parse() runs the same command and returns a dictionary. Compare them on show ip interface brief:

Interface              IP-Address      OK? Method Status                Protocol
Ethernet0/0            192.168.99.1    YES NVRAM  up                    up
Ethernet0/1            unassigned      YES NVRAM  administratively down down

! Command: dev.parse('show ip interface brief')        # Genie parser
{
  "interface": {
    "Ethernet0/0": {
      "interface_is_ok": "YES",
      "ip_address": "192.168.99.1",
      "method": "NVRAM",
      "protocol": "up",
      "status": "up"
    },
    "Ethernet0/1": {
      "ip_address": "unassigned",
      "status": "administratively down"
    }
  }
}

Look at the shape, because this is where Genie parts company with the TextFSM and ntc-templates world. TextFSM gives you a flat list of rows to loop over. Genie gives you a dictionary keyed by the thing itself, so you index straight in with result['interface']['Ethernet0/0']['status'], and every parser has a published schema, so the keys are a contract rather than something you discover by printing. If turning CLI output into structured data you can query is new to you, that contrast is the whole point. A health check becomes a comprehension: filter that dict for status == 'up' and you get back ['Ethernet0/0', 'Loopback0', 'Loopback1', 'Loopback14'].

Protocol state works the same way. show ip ospf neighbor parses into neighbours keyed by router ID:

! Command: dev.parse('show ip ospf neighbor')
{
  "interfaces": {
    "Ethernet0/0": {
      "neighbors": {
        "2.2.2.2": {
          "address": "192.168.99.2",
          "dead_time": "00:00:31",
          "priority": 1,
          "state": "FULL/BDR"
        },
        "3.3.3.3": {
          "address": "192.168.99.3",
          "state": "FULL/DR"
        }
      }
    }
  }
}

Now "assert every neighbour is FULL" is a real test, not a regex against a screen scrape. show version parses the same way, handing you version, uptime, chassis_sn and last_reload_reason as fields.

learn(): a whole feature, not one command

parse() is one command in, one dict out. learn() runs several show commands, merges the results, and returns a model of an entire feature: a snapshot of what a protocol is doing. r1.learn("ospf").info comes back structured by VRF, address family, instance and area, and inside each area sits the full link-state database broken out by LSA type:

! Command: dev.learn('ospf')  -> .info
{
  "vrf": { "default": { "address_family": { "ipv4": { "instance": { "1": {
    "areas": { "0.0.0.0": {
      "area_type": "normal",
      "database": { "lsa_types": { "1": {
        "lsa_type": 1,
        "lsas": { "1.1.1.1 1.1.1.1": {
          "adv_router": "1.1.1.1",
          "lsa_id": "1.1.1.1",
...

That is the LSDB as a dictionary, so you can assert about your topology directly against it. Only useful if you can read it, though, so if lsa_types keyed by number is not immediately obvious, get the difference between a Type 1 and a Type 5 LSA straight first.

learn('interface') is flatter and more immediately useful:

! Command: python -> {i: v.get('enabled') for i,v in .info.items()}
{
  "Ethernet0/0": true,
  "Ethernet0/1": false,
  "Loopback0": true,
  "Loopback14": true
}

The models are vendor-neutral by design: that same call returns the same structure on NX-OS or IOS-XR with the platform-specific parsing hidden underneath. That portability is the entire argument for models over per-command parsing in a mixed estate.

The killer feature: learn, change, learn, Diff

Here is the sequence that justifies the toolchain. Snapshot state, change something, snapshot again, diff the two.

from genie.utils.diff import Diff

pre = r1.learn("interface")
r1.configure(["interface Loopback14",
              " description CHANGED BY PYATS RES-0002",
              " shutdown"])
post = r1.learn("interface")

Before and after for that interface, straight out of the model:

! Command: python -> pre.info['Loopback14']
{ "description": "PingLabz RES-0014 pushed by netmiko",
  "enabled": true, "oper_status": "up" }

! Command: python -> post.info['Loopback14']
{ "description": "CHANGED BY PYATS RES-0002",
  "enabled": false, "oper_status": "down" }

The counter-noise trap

Now diff them naively and watch it fall over:

! Command: d = Diff(pre.info, post.info); d.findDiff(); print(d)
 Ethernet0/0:
  counters:
-  in_octets: 179470
+  in_octets: 191412
-  out_errors: 0
+  out_errors: 6
   rate:
-   in_rate: 1000
+   in_rate: 3000
 Loopback14:
- description: PingLabz RES-0014 pushed by netmiko
+ description: CHANGED BY PYATS RES-0002
- enabled: True
+ enabled: False

Your change is in there, buried under interface counters that moved because time passed. Octets, packet counts, spanning-tree bytes, computed rates: all of it climbs every second on any link carrying traffic, and with three OSPF speakers flooding hellos onto this segment there is no quiet moment. A pre/post test written against that raw diff fails one hundred percent of the time, for reasons unrelated to the change. This is the most common reason people try Diff() once and abandon it.

The fix is the exclude= argument: name the volatile keys and they are dropped from both sides before comparison. Same snapshots, same change, and now the output is only the change:

! Command: Diff(pre.info, post.info, exclude=['counters', 'accounting', 'rate', 'last_change', 'in_rate', 'out_rate'])
 Loopback14:
- description: PingLabz RES-0014 pushed by netmiko
+ description: CHANGED BY PYATS RES-0002
- enabled: True
+ enabled: False
- oper_status: up
+ oper_status: down

Note what those three lines are. description is the config you sent. enabled and oper_status are the device reporting the consequence: it went administratively down and the line protocol followed. Nothing you diff in a config file reports that second half.

The empty diff is the proof

Roll it back and diff pre against a fresh snapshot, same exclusions:

! Command: dev.configure(['interface Loopback14',' no shutdown',' description PingLabz RES-0014 pushed by netmiko'])
! Command: Diff(pre.info, rolled_back.info, exclude=[...])
<empty diff - operational state is back exactly where it started>

Empty. That is a stronger claim than it looks. You have not merely verified that the rollback commands were accepted, which is all a config comparison tells you. You have verified that measured operational state is identical to what you recorded before you touched anything. "I typed no shutdown and it did not error" and "the interface is up and behaving as before" are different statements, and only one ends the change window.

Configuration diffs versus operational diffs

This distinction decides which tool you reach for, and it is why serious shops run more than one.

NAPALM diffs configuration. pyATS Genie diffs operational state. Neither replaces the other, so use both: let NAPALM show you the candidate config diff and gate the commit on it, then let Genie prove the network still works afterwards. For all three compared side by side on these same devices, see the netmiko versus NAPALM versus pyATS breakdown.

Parsing switch tables

The same parsers work against Layer 2, where the ugliest column formats live. On the ioll2-xe switch, show interfaces status parses cleanly, and the parser normalises the CLI's Et0/0 into Ethernet0/0 so keys match across commands:

! Command: sw1.parse('show interfaces status')
{
  "interfaces": { "Ethernet0/0": {
    "duplex_code": "full",
    "port_speed": "auto",
    "status": "connected",
    "type": "10/100/1000BaseTX",
    "vlan": "1"
  } }
}

show errdisable recovery, a two-column wall of reasons, becomes booleans plus the timer. show vlan, three stacked tables in one output, comes back keyed by VLAN ID with member ports as a list:

! Command: sw1.parse('show vlan')
{
  "vlans": { "1": {
    "interfaces": ["Ethernet0/0", "Ethernet0/1", "..."],
    "name": "default", "said": 100001,
    "state": "active", "vlan_id": "1"
  } }
}

The important behaviour: where no parser exists for a command, Genie raises rather than returning something half-parsed. That is the correct failure mode. You want a loud exception, not a dictionary quietly missing the key your test asserts on.

What this was captured on

The switch between the CML external connector and the routers is what makes this zero-routing: the VM and all three routers share one broadcast domain, so the VM just ARPs for them. It also means the segment carries OSPF hellos continuously, which is why the counter noise above was so aggressive.

Gotchas

• Do not probe with a question mark. r1.execute("show aaa ?") raises SubCommandFailure('... Incomplete command') even though the help text prints on screen, because Unicon wants a settled prompt and context-sensitive help never gives it one. Test feature support by running the real command and catching SchemaEmptyParserError.
• The first connect to a freshly booted node can exceed 60 seconds and time out. A retry against the same node succeeded immediately, so retry before assuming the device is broken.
• Genie's schema is nested, not row-based. Coming from ntc-templates you will reflexively loop over rows. There are no rows. Index by key, and read the parser's schema page rather than guessing.
• Always exclude counters before you diff. counters, accounting, rate, in_rate, out_rate and last_change covers the interface model. Other models have their own volatile fields (anything with a timer or a hit count), so build the list from your first noisy diff.
• Diff the whole model, not just the object you touched. The point of a state diff is catching side effects you did not expect. Narrow it to Loopback14 first and you have thrown away the reason you ran it.
• Watch the credentials block. These routers run privilege 15 with no enable secret, so pyATS lands straight in exec mode. If yours need an enable password, it goes in the testbed credentials as a separate entry.

Key takeaways

• The install objection is retired. pip install "pyats[full]" ships cp313 wheels and installs clean on Python 3.13 with no compiler.
• Your first blocker is SSH, not pyATS. Modern OpenSSH will not negotiate with 2020-era Cisco images until KexAlgorithms +diffie-hellman-group14-sha1 and HostKeyAlgorithms +ssh-rsa are in ~/.ssh/config.
• parse() turns one command into a schema-backed dictionary. learn() merges several commands into a vendor-neutral model of a whole feature, which is what you snapshot.
• Diff() without exclude= is useless on a live link, because packet counters move between the two learns whether or not you changed anything.
• NAPALM diffs configuration, pyATS Genie diffs operational state. Only the state diff catches "the config applied but the interface never came up", which is why you run both.
• An empty diff after rollback is the strongest proof of a clean back-out there is, because it is measured rather than assumed.

Once you have a diff you trust, wrap it in AEtest so post-change validation runs as pass/fail in CI instead of a human squinting at output. The rest of the tooling that gets you from manual show commands to tested changes covers what sits around it, and running Python directly on IOS-XE is the same idea with no automation host in the middle.