Community ID: Correlate Zeek and Suricata Logs from One PCAP

Suricata flagged the SYN for a connection that later returned HTTP 200. Zeek recorded the same connection as a completed HTTP exchange. Both observations were correct in this lab.

The Suricata rule counted SYNs by source during a short burst. It did not classify the request body or distinguish the scanner from the ordinary client sharing that address. Zeek's HTTP log described what happened after the connection started.

This article joins those views using Community ID, then checks two ways the comparison can go wrong: different hash seeds and different byte-counting units. It uses the exact capture analyzed in the Zeek walkthrough and the Suricata EVE lab.

Compare the same input and configuration

Use the sibling articles to generate evidence/mixed.pcap and these output directories:

evidence/zeek-mixed/conn.log
evidence/zeek-mixed/dns.log
evidence/zeek-mixed/http.log
evidence/suricata-mixed/eve.json

Tested with: Zeek 9.0.0, Suricata 8.0.3. Both analyzed one 79-packet file from the same server-side veth capture, with checksum validation disabled for the host-side capture representation. Zeek's loaded-script manifest and Suricata's saved configuration/rule file define the analysis.

The traffic includes two DNS exchanges, two complete HTTP requests, and 25 bounded Nmap SYN probes. The Nmap hub covers those probes, while the packet analysis hub covers input and capture-point checks.

A tool comparison against different PCAPs would mix visibility differences with analyzer behavior. Preserve the input hash and check timestamps and tuples before interpreting a disagreement.

Keep each tool's identifiers in its own scope

Within the Zeek run, uid connects an HTTP or DNS record to its connection record. Within Suricata's EVE output, flow_id joins flow, alert, and protocol events. These fields are not interchangeable across tools, and a Zeek UID can change on a new analysis run.

Community ID provides a tuple-derived correlation value that both tools can emit. This test uses version 1, base64 output, and seed 0 on both sides. Zeek loads policy/protocols/conn/community-id-logging; Suricata enables community-id in its EVE configuration.

The Zeek Community ID policy reference and Suricata EVE documentation describe those settings.

An equal Community ID is a join candidate. Check time and endpoint tuples too, especially with reused tuples, NAT boundaries, or different capture locations. Here both tools read the same bytes, and the 29 connection tuples have distinct Community IDs.

Join the recovery request to its alert with Community ID

Save this as scripts/join-logs.py and run python3 scripts/join-logs.py from the lab directory:

#!/usr/bin/env python3
import argparse,json,pathlib
p=argparse.ArgumentParser();p.add_argument('--zeek',default='evidence/zeek-mixed');p.add_argument('--eve',default='evidence/suricata-mixed/eve.json');a=p.parse_args()
def read(path):return [json.loads(line) for line in pathlib.Path(path).read_text().splitlines()]
conn=read(pathlib.Path(a.zeek)/'conn.log');http=read(pathlib.Path(a.zeek)/'http.log');eve=read(a.eve)
flows={row['community_id']:row for row in eve if row['event_type']=='flow'}
matched=[row for row in conn if row.get('community_id') in flows]
print('Zeek connections:',len(conn));print('Suricata flows:',len(flows));print('Community ID matches:',len(matched))
for request in http:
 if request.get('uri')!='/recovery':continue
 connection=next(row for row in conn if row['uid']==request['uid']);cid=connection['community_id'];events=[row for row in eve if row.get('community_id')==cid]
 print(json.dumps({'community_id':cid,'zeek_uid':connection['uid'],'zeek_status':request['status_code'],'suricata_events':[{'type':row['event_type'],'flow_id':row['flow_id'],'pcap_cnt':row.get('pcap_cnt'),'alert_sid':row.get('alert',{}).get('signature_id'),'http_status':row.get('http',{}).get('status')} for row in events]},indent=2))

The saved seed-0 comparison found 29 Zeek connections, 29 Suricata flows, and 29 matching Community IDs. The recovery connection's correlation value was:

1:glsMIl1J8F9zhUAEpsK6xtr4Wx0=

Following it joined these observations:

• Zeek UID CHPWL61Q5CkyYFxi1g: GET /recovery, host app.test, HTTP status 200.
• Suricata flow ID 1006345699527183: alert SID 9900001 at frame 68.
• The same Suricata flow ID: HTTP /recovery status 200, logged at frame 76, plus the final flow record.

Those are values from the saved run; discover the IDs from your own logs rather than expecting the same local IDs after replay.

The alert's frame was a SYN from 192.0.2.10:41002 to 192.0.2.20:8080. The HTTP record later described the established connection. The client checked the expected response body and the server logged the request.

This supports “the SYN matched the configured source-rate condition, and the application request succeeded.” It does not support “the HTTP request was malicious” or “the sensor blocked the connection.” The exact lab rule and the offline processing mode explain why those stronger claims would be wrong.

Break the join without changing the capture

Keep Suricata at seed 0 and analyze the same PCAP with Zeek seed 1. Use a fresh output directory and the same image from the Zeek article (prefix docker with sudo if your user can't run Docker):

ZEEK_IMAGE='zeek/zeek@sha256:70733f4e540ba1608e37e00c6a93d009f79734262c9ec2ba09d90aa9abc16de5'
mkdir evidence/zeek-seed1

docker run --rm --network none --user "$(id -u):$(id -g)" \
  -v "$PWD:/work" -w /work/evidence/zeek-seed1 "$ZEEK_IMAGE" \
  zeek -C -r /work/evidence/mixed.pcap LogAscii::use_json=T \
  policy/misc/loaded-scripts policy/protocols/conn/community-id-logging \
  CommunityID::seed=1

python3 scripts/join-logs.py --zeek evidence/zeek-seed1

Zeek still produced 29 connection records, two DNS records, and two HTTP records. The connection states and recovery status were unchanged. The join count fell from 29 to zero because the hash seed differed.

A missing cross-tool match here was a configuration mismatch, not missing traffic. Return to the existing seed-0 output and the 29 joins are available again. In a larger investigation, verify Community ID settings before treating unmatched records as evidence of a sensor blind spot.

Compare record counts at the right level

For this input, Zeek produced 29 connection records, two DNS records, and two HTTP records. Suricata produced 29 flow records, four DNS events, two HTTP events, and 17 alerts, plus a stats event.

The connection and flow counts agreed in this controlled example. The DNS counts differed because Suricata logged requests and responses separately, while the selected Zeek DNS records summarized the two exchanges. Suricata's DNS objects used version 3 and included type: request or type: response.

Do not equate Zeek's trans_id with Suricata's tx_id. In this sample, the DNS wire ID 4097 appeared as Zeek trans_id: 4097 and Suricata dns.id: 4097; Suricata's tx_id was an analyzer transaction field. Use the query, wire ID, tuple, and time to check the exchange.

The 17 alerts are not 17 extra connections. They are events attached to flows already present in the 29-flow set. An empty-rule Suricata replay kept all 29 flow records and the protocol metadata while producing no alerts.

Compare bytes without inventing packet loss

The recovery connection had six packets in each direction in both tools. For the client-to-server direction, the saved records reported:

Zeek orig_bytes:                 61
Zeek orig_ip_bytes:             381
Suricata flow.bytes_toserver:   465

These are different accounting levels. In this Ethernet capture, Zeek's 61 application bytes are the HTTP request payload. Its IP byte total includes IP and transport headers. The Suricata flow byte total also includes the captured Ethernet headers.

The difference between 465 and 381 is 84 bytes: six packets times the 14-byte Ethernet header visible in this untagged veth capture. The reverse direction similarly had 456 Zeek IP bytes and 540 Suricata flow bytes.

The packet counts and the header-length difference were checked against the PCAP fields. VLAN tags, tunnels, or other link layers change that arithmetic, and similar field names do not make two counters comparable.

Use disagreement to choose the next check

For a failed join, compare the hash settings, tuple, timestamp, and capture location. For unequal DNS counts, compare what each record represents. For unequal byte totals, identify their accounting level before claiming packets disappeared. For an alert on a successful request, read the rule and the later application evidence.

The tools answer different parts of an investigation. This capture gives Zeek a connection/protocol timeline and gives Suricata a testable signature event on the same flow. Keep those observations attached to the raw packets and the analysis configuration, then make the narrowest claim they support.