> ## Content Index > Fetch the complete content index at: https://www.pinglabz.com/llms.txt > Use this file to discover other available public pages before exploring further. # Telegraf, gNMI and Cisco IOS XE: Check Support First, Then Poll Faster - URL: https://www.pinglabz.com/grafana-telegraf-gnmi-ios-xe/ - Published: 2026-11-03T03:32:30.000Z - Updated: 2026-11-03T03:58:03.000Z - Description: I tried to stream IOS XE interface counters into Prometheus with Telegraf and gNMI. The image had no gNMI, so here's how to check, what the errors mean, and what 10-second SNMP polling shows that 60-second polling hides. - Author: Jaime - Tags: Grafana, Monitoring, SNMP, Cisco IOS XE, Linux, Tools The usual pitch for streaming telemetry goes like this: SNMP polls every minute or so and smooths everything out, gNMI pushes a sample every 10 seconds, so switch to gNMI. Telegraf is a popular collector for that because one agent can subscribe over gNMI, poll SNMP and expose the result to Prometheus. I tried to build exactly that against two Cisco IOS XE 17.18.2 routers, and the gNMI half didn't work. This post covers why, what I ran instead, and what 10-second data actually shows you that 60-second data hides. This continues the [Prometheus and snmp\_exporter build](https://www.pinglabz.com/grafana-prometheus-snmp-exporter-ios-xe/) and the [syslog into Loki post](https://www.pinglabz.com/grafana-loki-syslog-ios-xe/). Same routers, same Prometheus, same Grafana 13.0.2. [![Lab diagram: Telegraf 1.40.3 at 10.66.20.44 first tried gNMI on TCP 9339 to wan-rtr1 and wan-rtr2 (IOS XE 17.18.2). The connection was refused because there is no gnmi-yang support, so that path is drawn dashed. Telegraf instead polls the routers over SNMPv3 on UDP 161 every 10 seconds, allowed by ACL ZBX-SNMP. It exposes the results on port 9273 for Prometheus, which also scrapes snmp_exporter (10.66.20.41) every 60 seconds. One Grafana panel compares Et0/1 inbound traffic at both rates: over 20 minutes the 10-second line peaks at about 42 Mb/s, while the 60-second line never goes above about 26 Mb/s. A tip explains the difference: a refused connection means the service is missing, while a timeout means an ACL or the path is dropping you.](https://storage.ghost.io/c/ff/28/ff28db4a-11e1-4835-b928-158974cf96c1/content/images/2026/11/diagram-topology-26.png)](https://storage.ghost.io/c/ff/28/ff28db4a-11e1-4835-b928-158974cf96c1/content/images/2026/11/diagram-topology-26.png?ref=pinglabz.com) gNMI on TCP 9339 was refused (no gnmi-yang), so Telegraf polls SNMPv3 every 10 s next to snmp\_exporter's 60 s. ## Check the device first gNMI on IOS XE depends on the programmability stack (the same one NETCONF and RESTCONF use). Before you write a single line of collector config, confirm the image has it: ``` wan-rtr1#show version | include Version Cisco IOS Software [IOSXE], Linux Software (X86_64BI_LINUX-ADVENTERPRISEK9-M), Version 17.18.2, RELEASE SOFTWARE (fc3) wan-rtr1#show gnmi-yang state show gnmi-yang state ^ % Invalid input detected at '^' marker. wan-rtr1#show netconf-yang status show netconf-yang status ^ % Invalid input detected at '^' marker. ``` The configuration side is the same story: `gnmi-yang` and `netconf-yang` are rejected in global config, and `show telemetry ietf subscription all` doesn't exist either. This build of IOS XE is a Linux-hosted software image without the YANG management process. Hardware platforms (Catalyst 8000, Catalyst 9000, ASR 1000) and the virtual Catalyst 8000V do ship it, so on those you would enable it with something like: ``` gnmi-yang gnmi-yang server gnmi-yang port 50052 ! or the TLS server on 9339 with a trustpoint: gnmi-yang secure-server gnmi-yang secure-trustpoint ``` I couldn't test that block here, so treat it as a pointer to the right commands, not a verified config. Check the programmability configuration guide for your release before you rely on port numbers or TLS options. ## The Telegraf config Telegraf 1.40.3 runs on a Linux host next to Prometheus. One config file has three parts: a gNMI input, an SNMP input, and a Prometheus output. Credentials come from environment variables in a file only root can read, never from the config itself. ``` [agent] interval = "10s" flush_interval = "10s" omit_hostname = true # gNMI dial-in [[inputs.gnmi]] addresses = ["10.66.20.11:9339", "10.66.20.12:9339"] username = "${GNMI_USER}" password = "${GNMI_PASS}" encoding = "json_ietf" redial = "30s" tls_enable = true insecure_skip_verify = true [[inputs.gnmi.subscription]] name = "ifcounters" origin = "openconfig" path = "/interfaces/interface/state/counters" subscription_mode = "sample" sample_interval = "10s" # SNMPv3 polling every 10s [[inputs.snmp]] agents = ["udp://10.66.20.11:161", "udp://10.66.20.12:161"] version = 3 sec_name = "${SNMP_USER}" sec_level = "authPriv" auth_protocol = "SHA" auth_password = "${SNMP_AUTH}" priv_protocol = "AES" priv_password = "${SNMP_PRIV}" timeout = "5s" retries = 1 [[inputs.snmp.field]] name = "sysName" oid = ".1.3.6.1.2.1.1.5.0" is_tag = true [[inputs.snmp.table]] name = "ifx" inherit_tags = ["sysName"] [[inputs.snmp.table.field]] name = "ifName" oid = ".1.3.6.1.2.1.31.1.1.1.1" is_tag = true [[inputs.snmp.table.field]] name = "ifHCInOctets" oid = ".1.3.6.1.2.1.31.1.1.1.6" [[inputs.snmp.table.field]] name = "ifHCOutOctets" oid = ".1.3.6.1.2.1.31.1.1.1.10" [[inputs.snmp.table.field]] name = "ifOperStatus" oid = ".1.3.6.1.2.1.2.2.1.8" [[outputs.prometheus_client]] listen = ":9273" metric_version = 2 ``` A few details that cost me a restart each: - The gNMI TLS switch is `tls_enable`. I first wrote `enable_tls`, and Telegraf refused to start at all: *configuration specified the fields \["enable\_tls"\], but they were not used*. Strict config checking is a good thing, but it means one typo in one plugin takes down every input. - Numeric OIDs avoid any MIB loading. The SNMP input works without MIB files on the collector, and the field names you choose become the metric names. - Since 1.38, Telegraf handles environment variables strictly and logs a warning about it on every start. If a `${VAR}` isn't set, it's an error, not an empty string. ## What the logs said With the config loaded, both inputs failed, for different reasons: ``` E! [inputs.gnmi] Error in plugin: failed to setup subscription: rpc error: code = Unavailable desc = connection error: desc = "transport: Error while dialing: dial tcp 10.66.20.11:9339: connect: connection refused" E! [inputs.snmp] Error in plugin: agent udp://10.66.20.11:161: performing get on field sysName: request timeout (after 1 retries): max retries exceeded ``` The gNMI error is the expected one: nothing listens on 9339 because the image has no gNMI server. A connection refused tells you the IP path is fine and the service is missing. A timeout here would point at an ACL or routing instead. The SNMP timeout was mine. The routers only answer SNMP from addresses in a standard ACL attached to the SNMPv3 group, and the new collector wasn't in it. SNMPv3 doesn't send an error back for a denied source; it just drops the request, so it looks exactly like a dead agent. The fix on each router: ``` ip access-list standard ZBX-SNMP 30 permit 10.66.20.44 ! snmp-server group ZBX-RO v3 priv read ZBX-VIEW access ZBX-SNMP ``` On the next poll the counters showed up on the Prometheus endpoint: ``` $ curl -s 10.66.20.44:9273/metrics | grep ^ifx_ifHCInOctets ifx_ifHCInOctets{agent_host="10.66.20.11",ifName="Et0/0",sysName="wan-rtr1.lab"} 9.3381566e+07 ifx_ifHCInOctets{agent_host="10.66.20.11",ifName="Et0/1",sysName="wan-rtr1.lab"} 3.452056894e+09 ifx_ifHCInOctets{agent_host="10.66.20.11",ifName="Et0/2",sysName="wan-rtr1.lab"} 121624 ``` ## Scraping Telegraf from Prometheus Telegraf's `prometheus_client` output is just another target. Match the scrape interval to the agent interval, or you'll either duplicate samples or throw half of them away: ``` - job_name: telegraf scrape_interval: 10s static_configs: - targets: ["10.66.20.44:9273"] ``` ## 10 seconds vs 60 seconds The point of all this was resolution, so the dashboard puts the same counter from both pipelines on one graph: wan-rtr1 Et0/1 inbound, once from Telegraf every 10 seconds and once from snmp\_exporter every 60 seconds. ``` # Telegraf, 10s rate(ifx_ifHCInOctets{job="telegraf",sysName="wan-rtr1.lab",ifName="Et0/1"}[30s]) * 8 # snmp_exporter, 60s rate(ifHCInOctets{job="snmp-cisco-router",instance="10.66.20.11",ifName="Et0/1"}[2m]) * 8 ``` The rate windows differ on purpose. `rate()` needs at least two samples inside the window, so the window has to be at least two scrape intervals: 30 seconds covers three Telegraf samples, and 2 minutes covers two exporter samples. ![Grafana panel comparing 10-second Telegraf SNMP polling with 60-second snmp_exporter polling on the same IOS XE interface](https://storage.ghost.io/c/ff/28/ff28db4a-11e1-4835-b928-158974cf96c1/content/images/2026/11/g3-board.png) The traffic on that link comes in bursts of one to four minutes with short pauses between them. In this 20-minute window the 10-second line peaks at about 42 Mb/s. The 60-second line never goes above about 26 Mb/s, and while that 42 Mb/s burst was happening it still showed about 14 Mb/s, because its last sample was taken before the burst began. Slow polling doesn't just make the graph smoother. It can put a peak in the wrong minute or hide it completely. If you size links or alert on utilization, the 60-second view can show a link with plenty of headroom while it is actually near its peak in short bursts. One Grafana gotcha: my first screenshot of this panel showed the Telegraf line with one point per minute, even though Prometheus had a sample every 10 seconds. The query step was following the data source's default interval. Setting the panel's **Min interval** (query options) to `10s` fixed it. There's a cost. Six times the samples means six times the SNMP load on the router's CPU and six times the storage in Prometheus for those series. Fast polling on a few uplinks is fine. Fast polling every interface on a few hundred switches is the problem streaming telemetry was designed to solve, which is why it's worth checking platform support before you plan around it. ## When gNMI is available On a platform that has the programmability stack, keep the same pipeline and remove the fast SNMP input. The gNMI subscription above gives you the same interface counters as fields of an `ifcounters` measurement, tagged with the interface name, and the Prometheus side doesn't change. Keep slow SNMP running for things the OpenConfig models on your platform don't cover. Run `show gnmi-yang state` on the device first, check it says the server is enabled, and look for a connection from the collector before debugging the Telegraf side. ## Takeaways - Check the device supports gNMI before building the collector. On this IOS XE image the commands don't exist at all. - Connection refused means the service is missing. A timeout means something in the path, often an SNMP ACL, is dropping you. - Telegraf's SNMP input at 10 seconds is a reasonable stopgap for a few important interfaces. - Set `rate()` windows to at least two scrape intervals of the job you're querying. Now live: [Grafana alerting with contact points, policies and silences](https://www.pinglabz.com/grafana-alerting-email-ios-xe/).