Ansible Inventory for Network Engineers: Groups, group_vars and host_vars by Site and Role

A three-router inventory can be one flat list. Six devices across three sites can't. Some settings follow the site (which NTP server is closest, what the SNMP location says) and some follow the role (edge routers send logs to the security collector). The inventory has to describe both and let Ansible combine them per device.

This is part 2 of the PingLabz Ansible for Network Engineers series. Part 1, Ansible Network Lab Setup, got three HQ routers answering a playbook. This post grows the lab to three sites and six devices, builds a site-and-role inventory, and runs into a precedence rule I hadn't expected: a site value silently beat a role value.

I built this in my home lab running Cisco IOS XE 17.18.2 (five routers and one L2 switch), ansible-core 2.21.5 and cisco.ios 11.6.0. All the output is from real runs. Passwords are placeholders.

The lab

What each device should end up with:

• NTP: HQ uses 10.66.0.123. Both branches use a regional server, 10.66.20.123.
• Syslog: everything logs to 10.66.0.51, except BR1, which has its own local collector at 10.66.21.51. Every edge router also has to log to the security collector at 10.66.0.52.
• SNMP location: per site, with one override for hq-edge1, which sits in a different rack.

Two axes, one inventory

Use two parent groups that don't overlap in meaning: sites and roles. Every device appears once under each.

all:
  children:
    sites:
      children:
        hq:
          hosts:
            hq-core1: {ansible_host: 10.66.10.11}
            hq-core2: {ansible_host: 10.66.10.12}
            hq-edge1: {ansible_host: 10.66.10.13}
        branches:
          children:
            br1:
              hosts:
                br1-edge1: {ansible_host: 10.66.10.21}
                br1-acc1: {ansible_host: 10.66.10.22}
            br2:
              hosts:
                br2-edge1: {ansible_host: 10.66.10.31}
    roles:
      children:
        core:
          hosts:
            hq-core1:
            hq-core2:
        edge:
          hosts:
            hq-edge1:
            br1-edge1:
            br2-edge1:
        access:
          hosts:
            br1-acc1:

Only the site groups set ansible_host. The role groups list host names only, so each IP lives in exactly one place. branches is a group of groups, which gives you one place for anything that applies to every branch.

ansible-inventory --graph is the fastest way to check that the tree is what you meant:

$ ansible-inventory -i site-inv --graph
@all:
  |--@ungrouped:
  |--@sites:
  |  |--@hq:
  |  |  |--hq-core1
  |  |  |--hq-core2
  |  |  |--hq-edge1
  |  |--@branches:
  |  |  |--@br1:
  |  |  |  |--br1-edge1
  |  |  |  |--br1-acc1
  |  |  |--@br2:
  |  |  |  |--br2-edge1
  |--@roles:
  |  |--@core:
  |  |  |--hq-core1
  |  |  |--hq-core2
  |  |--@edge:
  |  |  |--hq-edge1
  |  |  |--br1-edge1
  |  |  |--br2-edge1
  |  |--@access:
  |  |  |--br1-acc1

Variables: group_vars and host_vars

Keep the data out of hosts.yml. Each group gets a folder under group_vars, and a host that needs an exception gets a file under host_vars:

site-inv/
├── hosts.yml
├── group_vars/
│   ├── all/
│   │   ├── connection.yml   network_cli, ios, become
│   │   ├── site.yml         defaults: ntp, syslog, snmp contact
│   │   └── vault.yml        encrypted passwords
│   ├── hq/site.yml          site_code, site_location
│   ├── branches/site.yml    ntp_server: 10.66.20.123
│   ├── br1/site.yml         site_code, site_location, syslog_server: 10.66.21.51
│   ├── br2/site.yml         site_code, site_location
│   ├── core/role.yml        device_role: core
│   ├── edge/role.yml        device_role: edge, syslog_server: 10.66.0.52
│   └── access/role.yml      device_role: access
└── host_vars/
    └── hq-edge1.yml         site_location override

A folder per group, rather than a single hq.yml, lets you split connection settings, site data and secrets into separate files, and only one of them has to be vault-encrypted. The connection settings from part 1 move into group_vars/all/connection.yml unchanged:

ansible_connection: ansible.netcommon.network_cli
ansible_network_os: cisco.ios.ios
ansible_user: netauto
ansible_password: "{{ vault_ios_password }}"
ansible_become: true
ansible_become_method: enable
ansible_become_password: "{{ vault_enable_password }}"

Check the merged result before you push anything

This read-only playbook prints what each device actually ends up with after Ansible merges every layer:

- name: Show the variables each device ends up with
  hosts: all
  gather_facts: false
  tasks:
    - name: Merged view per host
      ansible.builtin.debug:
        msg: "{{ site_code }}/{{ device_role }} ntp={{ ntp_server }} syslog={{ syslog_server }} loc={{ site_location }}"

ok: [hq-edge1]  => "HQ/edge ntp=10.66.0.123 syslog=10.66.0.52 loc=HQ data room, rack B1 (ISP handoff)"
ok: [br1-edge1] => "BR1/edge ntp=10.66.20.123 syslog=10.66.21.51 loc=Branch 1 comms closet"
ok: [hq-core2]  => "HQ/core ntp=10.66.0.123 syslog=10.66.0.51 loc=HQ data room, rack A3"
ok: [hq-core1]  => "HQ/core ntp=10.66.0.123 syslog=10.66.0.51 loc=HQ data room, rack A3"
ok: [br1-acc1]  => "BR1/access ntp=10.66.20.123 syslog=10.66.21.51 loc=Branch 1 comms closet"
ok: [br2-edge1] => "BR2/edge ntp=10.66.20.123 syslog=10.66.0.52 loc=Branch 2 comms closet"

Most of it is right:

• The branch NTP from branches reached both BR1 and BR2.
• The host_vars override gave hq-edge1 its own rack.
• hq-edge1 and br2-edge1 picked up the edge syslog collector.

But br1-edge1 is an edge router and got 10.66.21.51, not 10.66.0.52. The site value won and the role value lost, and nothing printed a warning.

Why the site beat the role

When the same variable is defined in two groups, Ansible doesn't look at which group is more important to you. It sorts the groups by depth and lets the deeper (more specific) group win:

• br1 is all → sites → branches → br1, depth 3.
• edge is all → roles → edge, depth 2.

Depth 3 beats depth 2. For br2-edge1 the only competitor was all, so edge won there. That's why one edge router was wrong and the other two were right.

ansible-inventory --host shows the final value and saves you guessing:

$ ansible-inventory -i site-inv --host br1-edge1
{
    "ansible_host": "10.66.10.21",
    "device_role": "edge",
    "ntp_server": "10.66.20.123",
    "site_code": "BR1",
    "site_location": "Branch 1 comms closet",
    "snmp_contact": "noc@example.net",
    "syslog_server": "10.66.21.51",
    "vault_enable_password": "<ENABLE_SECRET>",
    "vault_ios_password": "<NETAUTO_PASSWORD>"
    ...
}

Look at the last two lines. ansible-inventory --host decrypts vault variables and prints them in clear text. In my run the real lab password was right there in the terminal (I've replaced it with placeholders here). Never paste this output into a ticket or a chat.

ansible_group_priority doesn't fix it

The obvious fix is ansible_group_priority. I tried it two ways. Set in group_vars/edge/role.yml, it did nothing. Ansible only reads it from the inventory source. Moved into hosts.yml under edge: vars::

$ ansible -i site-inv br1-edge1 -m debug -a var=syslog_server
br1-edge1 | SUCCESS => {
    "syslog_server": "10.66.21.51"
}

Still the site value. Group priority only breaks ties between groups at the same depth. Depth is compared first, so a priority of 10 on a depth-2 group never beats a depth-3 group.

The fix: don't let site and role own the same variable

The real problem was the design: two independent groups both claimed syslog_server. The edge requirement isn't a replacement for the site syslog server. It's an extra destination. So give it its own variable:

# group_vars/all/site.yml
role_syslog_servers: []

# group_vars/edge/role.yml
device_role: edge
role_syslog_servers:
  - 10.66.0.52

Site groups own syslog_server, role groups own role_syslog_servers, and nothing overlaps. The rule I follow now: every variable has one owning axis. If the same key shows up under a site group and a role group, expect a precedence problem.

Push the baseline from the inventory

---
- name: Site and role baseline
  hosts: all
  gather_facts: false
  tasks:
    - name: NTP, syslog and SNMP identity from inventory data
      cisco.ios.ios_config:
        lines: "{{ base_lines + role_lines }}"
      vars:
        base_lines:
          - "ntp server {{ ntp_server }}"
          - "logging host {{ syslog_server }}"
          - "snmp-server location {{ site_location }}"
          - "snmp-server contact {{ snmp_contact }}"
        role_lines: "{{ role_syslog_servers | map('regex_replace', '^', 'logging host ') | list }}"

Target groups with patterns

The two axes pay off when you choose what to touch. edge:&branches means "in edge AND in branches":

$ ansible -i site-inv "edge:&branches" --list-hosts
  hosts (2):
    br1-edge1
    br2-edge1

Other patterns you'll use: branches:!access (branches except switches), core:edge (either), hq:&edge (one device today, more later). Check the canary pair first:

$ ansible-playbook -i site-inv playbooks/11-site-baseline.yml --limit "edge:&branches" --check

TASK [NTP, syslog and SNMP identity from inventory data] ***********************
changed: [br2-edge1]
changed: [br1-edge1]

PLAY RECAP *********************************************************************
br1-edge1                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0
br2-edge1                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0

Then the whole estate, and a second run to prove it's idempotent:

$ ansible-playbook -i site-inv playbooks/11-site-baseline.yml
...
PLAY RECAP *********************************************************************
br1-acc1                   : ok=1    changed=1    unreachable=0    failed=0    skipped=0
br1-edge1                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0
br2-edge1                  : ok=1    changed=1    unreachable=0    failed=0    skipped=0
hq-core1                   : ok=1    changed=1    unreachable=0    failed=0    skipped=0
hq-core2                   : ok=1    changed=1    unreachable=0    failed=0    skipped=0
hq-edge1                   : ok=1    changed=1    unreachable=0    failed=0    skipped=0

$ ansible-playbook -i site-inv playbooks/11-site-baseline.yml
...
PLAY RECAP *********************************************************************
br1-acc1                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0
br1-edge1                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0
br2-edge1                  : ok=1    changed=0    unreachable=0    failed=0    skipped=0
hq-core1                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0
hq-core2                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0
hq-edge1                   : ok=1    changed=0    unreachable=0    failed=0    skipped=0

Verify on the devices

br1-edge1:
    logging host 10.66.21.51
    logging host 10.66.0.52
    snmp-server location Branch 1 comms closet
    snmp-server contact noc@example.net
    ntp server 10.66.20.123

br2-edge1:
    logging host 10.66.0.51
    logging host 10.66.0.52
    snmp-server location Branch 2 comms closet
    snmp-server contact noc@example.net
    ntp server 10.66.20.123

hq-core1:
    logging host 10.66.0.51
    snmp-server location HQ data room, rack A3
    snmp-server contact noc@example.net
    ntp server 10.66.0.123

br1-edge1 now logs to its local collector and the security collector. The core router has no security collector line, because it isn't in edge.

Troubleshooting inventory problems

Key takeaways

• Model networks on two axes: site (where it is) and role (what it does). Every device belongs to one group in each.
• Set ansible_host in one place only, and keep data in group_vars/<group>/ folders.
• Deeper groups win. Depth is compared before ansible_group_priority, so priority can't rescue a cross-axis conflict.
• Give each variable one owning axis. When a role needs to add something, use a separate variable and merge them in the task.
• ansible-inventory --graph checks the structure and --host checks the values. And --host prints vault secrets in clear text.
• Use host patterns like edge:&branches with --check as a built-in canary.

Syslog and SNMP are a good first payload because you can verify them by eye. For the operations side, see Syslog Timestamps Out of Order: ASA vs IOS XE and SNMP RW Community Abuse on IOS XE. If YAML still feels unnatural, JSON vs XML vs YAML for Network Engineers is a quick primer. The next posts in the series turn these inventory variables into Jinja2 templates and resource-module data.