Snmpbulk works, but snmp_exporter does not

[remchuk]

Host operating system: output of uname -a

linux 4.18.0-372.16.1.el8_6.x86_64 #1 SMP Tue Jun 28 03:02:21 EDT 2022 x86_64 x86_64 x86_64 GNU/Linux

snmp_exporter version: output of snmp_exporter -version

build user: root@90ba0aabb239
build date: 20240511 - 11:16:35
go version: 1.22.3
platform: linux/amd64
tags: unknown

What device/snmpwalk OID are you using?

1.3.6.1.2.1.47.1.1.1.1.7 - entPhysicalName
on cisco switch, one NXOS and one is IOS-XE

If this is a new device, please link to the MIB(s).

What did you do that produced an error?

Just used the SNMP ui with the following generator.yml

auths:
    public_v2:
         community: public
         version: 2
    vrf:
         community: vrf
         version: 2

modules:
  switch:
    walk:
      - 1.3.6.1.2.1.47.1.1.1.7
    retries: 2
    timeout: 3s

What did you expect to see?

To receive metrics

What did you see instead?

An error has occurred serving metrics:
error collecting metic Desc{fqName: "snmp_error", help: "Error scrapping target", constLabels: {module="switch"}, variableLabels: {}}: error walking target <target-ip/hostname>: request timeout (after 2 retries)

When running tcpdump on my PC I see that :

17:23:42.326221 IP <my pc>.<big random port> > <cisco switch hostname>.snmp: GetBulk(36) N=0 M=25 47.1.1.1.7.300010563
17:23:42.326221 IP <my pc>.<big random port> > <cisco switch hostname>.snmp: GetResponse(1450) 47.1.1.1.1.7.300010564=<some long reponse>

17:23:42.326221 IP <my pc>.<big random port> > <cisco switch hostname>.snmp: GetBulk(36) N=0 M=25 47.1.1.1.7.300017603
17:23:45.326690 IP <my pc>.<big random port> > <cisco switch hostname>.snmp: GetBulk(36) N=0 M=25 47.1.1.1.7.300017603
17:23:48.328549 IP <my pc>.<big random port> > <cisco switch hostname>.snmp: GetBulk(36) N=0 M=25 47.1.1.1.7.300017603

[remchuk]

After some digging into the snmp on the switch, it seems like I recieve some errors,

"header_ifXEntry:request failed for ifindex=0" "vsan_intf_get_ifXEntry:cmi_req_resp failed" "fv_intf_get_ifXEntry:cmi_req_resp failed" "header_ciefIfInterfaceEntryTable: request failed for ifindex=0" "nve_intf_get_ifX_info : nve_get_next_port Ifindex failed" "nve_intf_get_ifXEntry: nve_mib_get_ifX_info Ifindex failed"

[bastischubert]

Hi! Have you tried to increase the timeout (3s in that config vs 6 sec from tcpdump)?

[remchuk]

Hi! Have you tried to increase the timeout (3s in that config vs 6 sec from tcpdump)?

Yes sir

[bastischubert]

@remchuk
What timeout did you use instead?
Can you use --snmp.debug-packets --log.level=debug with snmp_exporter and send the output as well?