Netmiko vs NAPALM vs pyATS: Three Ways to Diff a Cisco Change

You push a change to three routers, the script scrolls past without a single % line, and everything looks fine. Now answer the only question that matters at 2am: did that run actually change anything, and is the network doing what you intended? Most automation arguments about Netmiko, NAPALM and pyATS are really arguments about that one question, because each of the three answers it at a completely different layer.

This is not a feature table copied from three README files. All three tools were run against the same three routers, on the same segment, on the same afternoon, from the same Debian box. That is what makes the comparison worth reading, and it is the reason this sits in the middle of how the Cisco automation stack actually fits together rather than off to one side. The lab is a CML topology called VM Automation Edge: R1, R2 and R3 are iol-xe nodes running IOS-XE 17.18.2 with OSPF 1 area 0, plus an ioll2-xe switch, all bridged onto 192.168.99.0/24 with a Debian 13 VM sitting on the same broadcast domain at 192.168.99.100.

Short version of the finding: Netmiko answers at the session layer, NAPALM answers at the configuration layer, and pyATS Genie answers at the operational state layer. They are three rungs on a ladder, not three competitors.

Netmiko: the session layer

Netmiko is an SSH session with the sharp edges filed off. It logs in, disables the pager, sends what you tell it to send, and hands you back a string. That is genuinely useful, and for reading a device it is all you need. Push a loopback with send_config_set() and you get the whole config session echoed back:

! Command: conn.send_config_set(config_commands)
configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#interface Loopback14
R1(config-if)# description PingLabz RES-0014 pushed by netmiko
R1(config-if)# ip address 14.14.14.1 255.255.255.255
R1(config-if)#end
R1#

The absence of a % line is how you know every command was accepted. Now run the exact same list a second time, with nothing changed in between:

! Command: conn.send_config_set(config_commands)   # second, identical run
configure terminal
Enter configuration commands, one per line.  End with CNTL/Z.
R1(config)#interface Loopback14
R1(config-if)# description PingLabz RES-0014 pushed by netmiko
R1(config-if)# ip address 14.14.14.1 255.255.255.255
R1(config-if)#end
R1#

Byte for byte the same output. IOS re-accepted every line, cheerfully, because the IOS CLI is imperative and does not care that the interface already exists. The return value of send_config_set() tells you precisely nothing about whether the device moved. Netmiko has no concept of desired state, so the only way to prove "no change" is to grab show running-config before and after and diff it yourself with difflib:

! Command: difflib.unified_diff(show run before 2nd push, show run after 2nd push)
<empty diff - running-config is byte-for-byte identical>

For contrast, that same diff after the first push is not empty at all:

--- running-config BEFORE 1st run
+++ running-config AFTER 1st run
@@ -20,2 +20,5 @@
 ip ospf 1 area 0
+interface Loopback14
+ description PingLabz RES-0014 pushed by netmiko
+ ip address 14.14.14.1 255.255.255.255
 interface Ethernet0/0

So Netmiko can tell you what changed, but only because you built the diff engine in Python and pointed it at scraped text. That distinction is the whole reason to reach for driving Cisco SSH sessions from Python one command at a time when you need the flexibility, and to reach for something else when you need a guarantee. Netmiko does have use_textfsm=True, which turns the CLI table into a list of dicts and gets you out of regex territory (the same reason structured data beats screen scraping everywhere else in automation), but parsing output is not the same thing as knowing whether you changed the box.

NAPALM: the configuration layer

NAPALM moves the question one layer up. You hand it a candidate configuration, and before anything is applied it asks the device what the difference would be. Driven from a Nornir inventory with napalm_configure(..., dry_run=True), that is this:

napalm_configure DRY RUN********************************************************
* R1 ** changed : True *********************************************************
vvvv napalm_configure DRY RUN ** changed : True vvvvvvvvvvvvvvvvvvvvvvvvvvvvvvvv INFO
+interface Loopback1
+ description NORNIR-MANAGED pinglabz-lab
+ ip address 10.99.1.1 255.255.255.0
+ ip ospf 1 area 0
+snmp-server location PingLabz-Lab-Rack1
+snmp-server contact netops@pinglabz.com
^^^^ END napalm_configure DRY RUN ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

The important detail is where that diff came from. It was not computed in Python. NAPALM copied the candidate to unix:candidate_config over SCP and ran show archive config incremental-diffs on the router. The device generated its own diff. That is why it is trustworthy, and it is also why archive has to be enabled on the box before any of this works.

Commit it (dry_run=False) and all three hosts come back changed=True with the same block of + lines. Then run the identical task again:

napalm_configure RE-RUN (identical input)***************************************
* R1 ** changed : False ********************************************************
vvvv napalm_configure RE-RUN (identical input) ** changed : False vvvvvvvvvvvvvv INFO
^^^^ END napalm_configure RE-RUN (identical input) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

! Command: python -> (result[host][0].changed, len(result[host][0].diff))
R1   changed=False  diff_len=0  diff=''
R2   changed=False  diff_len=0  diff=''
R3   changed=False  diff_len=0  diff=''

Nothing between the vvvv and ^^^^ markers, because there was nothing to say. NAPALM asked the device for the incremental diff, got an empty string back, and skipped the commit entirely. That is real idempotence, and note how different it is from the Netmiko result: not "the commands were re-accepted without error", but "no configuration commands were sent at all". Same three routers, same segment, same day. If you have ever compared how declarative config management tools differ from a script that just runs commands, this is that argument reduced to two evidence files.

The other thing you get for free is a rendered-per-host inventory. snmp_location was defined once on the ios_edge group and resolved on all three devices, while each router got its own mgmt_loopback address, which is the practical case for keeping a real inventory instead of a list of IP addresses.

pyATS and Genie: the operational state layer

NAPALM will happily tell you a change was applied. It will not tell you the interface came up. That is a different question, and it is the one pyATS answers.

Genie's learn() is not a parser for one command. It runs several show commands and merges them into a feature model, so dev.learn('interface') gives you what the device is currently doing. Snapshot before a change:

! Command: pre = dev.learn('interface')
! Command: python -> pre.info['Loopback14']
{
  "bandwidth": 8000000,
  "description": "PingLabz RES-0014 pushed by netmiko",
  "enabled": true,
  "oper_status": "up"
}

Change the description and shut the interface, snapshot again, and feed both into Diff(). The naive version is a mess, and that mess is the most useful thing in this whole article:

! Command: d = Diff(pre.info, post.info); d.findDiff(); print(d)
 Ethernet0/0:
  counters:
-  in_octets: 179470
+  in_octets: 191412
-  in_pkts: 2396
+  in_pkts: 2585
-  out_octets: 328888
+  out_octets: 366986
 Loopback14:
- description: PingLabz RES-0014 pushed by netmiko
+ description: CHANGED BY PYATS RES-0002
- enabled: True
+ enabled: False

The change you made is in there, buried under interface counters that move every second (three OSPF speakers on one segment guarantee traffic). A pre and post test written on that raw diff fails one hundred percent of the time for reasons that have nothing to do with the change under test. Exclude the volatile keys and you get the money shot:

! Command: Diff(pre.info, post.info, exclude=['counters', 'accounting', 'rate', 'last_change', 'in_rate', 'out_rate'])
 Loopback14:
- description: PingLabz RES-0014 pushed by netmiko
+ description: CHANGED BY PYATS RES-0002
- enabled: True
+ enabled: False
- oper_status: up
+ oper_status: down

oper_status is the field neither of the other two tools has. Netmiko diffs config text you scraped. NAPALM diffs a candidate configuration. Only pyATS catches "the config applied cleanly but the interface never came up", because it is comparing measured state, not intent.

Then roll the change back and run the same diff a third time:

! Command: Diff(pre.info, rolled_back.info, exclude=['counters', 'accounting', 'rate', 'last_change', 'in_rate', 'out_rate'])
<empty diff - operational state is back exactly where it started>

An empty diff there is not a boring result. It is the proof that the rollback was complete, measured against the state you actually recorded rather than an assumption that no shutdown did what it says. That is the shape of every useful pre and post change validation test: learn, change, learn, diff, and expect empty when you undo it.

The three side by side

What this was captured on

One CML lab, one L2 segment, no routing between the automation host and the devices. R1, R2 and R3 are iol-xe on IOS-XE 17.18.2 at 192.168.99.1-3, SW1 is ioll2-xe at .4, and the Debian 13 VM sits at 192.168.99.100 on an interface bridged into CML. Login is admin at privilege 15 with no enable secret, which is why find_prompt() returns R1# and not R1>. Netmiko 4.5.0 was the Debian system package on Python 3.13.5; Nornir 3.5.0, nornir_napalm 0.5.0, NAPALM 5.1.0 and pyATS 26.6 all installed from wheels into a venv on the same interpreter.

Gotchas that stop each tool before it starts

• NAPALM on IOL will not even reach the diff. iol-xe has no flash: filesystem (show file systems lists nvram:, unix:, disk0:, disk1:, system:), and the napalm-ios driver defaults dest_file_system="flash:". Every load_merge_candidate() fails until the inventory sets optional_args: {dest_file_system: "unix:"}. You also need ip scp server enable for the transfer and archive with path unix:archive for the diff engine itself.
• pyATS dies on SSH negotiation, not on pyATS. A 2026 OpenSSH client will not talk to a 2020-era Cisco image. The fix is a ~/.ssh/config stanza for 192.168.99.* adding KexAlgorithms +diffie-hellman-group14-sha1, HostKeyAlgorithms +ssh-rsa and PubkeyAcceptedAlgorithms +ssh-rsa. It applies to Netmiko and NAPALM too.
• TextFSM fails silently. The ntc-templates template for show ip interface brief names its columns intf, ipaddr, status and proto, not interface and ip_address. Guess wrong and every row prints None with no exception raised. Print result[0].keys() once before you write the loop.
• Netmiko 4.x defaults to fast_cli=True. Over a bridged path to IOL that trimmed delay factor causes truncated reads. Every capture here used fast_cli=False.
• Genie Diff() without exclude= is unusable on a live segment. Counters, accounting and rate keys move constantly. Exclude them or your test never passes.
• napalm_get forwards every kwarg to every getter. napalm_get(getters=['config','interfaces'], retrieve='running') raises TypeError: IOSDriver.get_interfaces() got an unexpected keyword argument 'retrieve'. Getters with options need their own task.

How the three fit in one pipeline

Nobody picks one. A real change workflow uses NAPALM (usually through Nornir) to render intent from an inventory and push it, so the device generates the diff and a re-run is a no-op. It uses pyATS to learn the relevant feature model before and after and assert the operational diff contains exactly the intended change and nothing else. And it keeps Netmiko around for the awkward command a structured tool will not do: a vendor-specific show nothing has a parser for, a password recovery step, a box that only speaks Telnet. The same instinct applies to running Python directly on the switch when the automation host cannot reach it at all.

If you want the honest test of whether your pipeline is doing anything real, run it twice. A pipeline that cannot tell you the second run changed nothing is not automation, it is a fast way to type.

Key Takeaways

• Netmiko has no concept of desired state. An identical second send_config_set() produces an identical happy session echo, and only a difflib diff of show running-config proves the box did not move.
• NAPALM's diff is generated by the device (show archive config incremental-diffs), so an identical second run returns changed=False with diff='' and sends no commands at all.
• Genie learn() plus Diff() compares what the network is doing, including oper_status, which is the only one of the three that catches a change that applied but did not take effect.
• An empty diff after a rollback is a result, not a non-event. It is the proof the rollback was complete.
• Each tool has one setup detail that will stop you cold: fast_cli and TextFSM key names, dest_file_system: "unix:", and the legacy SSH algorithm stanza.
• Use all three. Push with NAPALM or Nornir, validate with pyATS, keep Netmiko for the one-off, and follow the rest of the Cisco automation series for the layers around them.