system cpu usage percent reported as 4294967295

[bilbof]

Describe the bug

We are using this exporter and are seeing invalid data returned by the /nitro/v1/stat/system endpoint.

JSON provided by the endpoint /nitro/v1/stat/system (redacted for brevity):

{ "errorcode": 0, "message": "Done", "severity": "NONE", "system": { "cpuusage": "4294967295", "rescpuusage": "9", "slavecpuusage": "0", "mastercpuusage": "9", "numcpus": "3", "memusagepcnt": 17.192564, "memuseinmb": "897", "addimgmtcpuusagepcnt": 0.000000, "mgmtcpu0usagepcnt": 1.000000, "mgmtcpuusagepcnt": 1.000000, "pktcpuusagepcnt": 8.600000, "cpuusagepcnt": 4294967295.000000, "rescpuusagepcnt": 4294967295.000000 } }

Note that system.cpuusagepcnt is reported as 4294967295.000000, (2**32 - 1). I suspect there is a divide by zero or similar in the stats code. It would be easier to graph CPU usage if this case could be handled as zero rather than infinity.

Forgive me if this is the wrong place to raise this bug.

[aroraharsh23]

May i know if you are scraping VPX, CPX or MPX ? And which version?

[bilbof]

ADC VPX. As far as I'm aware it's the latest release we're running, let me check.

Forgot to mention that in the GUI we are seeing these stats:

Packet CPU usage (%) 6.40
Management CPU usage (%) 1.90

[aroraharsh23]

i verified on

show version
NetScaler NS13.1: Build 24.29.nc, Date: Apr 12 2022, 12:22:50 (64-bit)
Done

where the same command gave output:
curl -k -u nitro/v1/stat/system

############
{ "errorcode": 0, "message": "Done", "severity": "NONE", "system": { "voltagev12n": 0.000000, "voltagev5n": 0.000000, "cpuusage": "1", "rescpuusage": "4294967295", "slavecpuusage": "4294967295", "mastercpuusage": "4294967295", "auxvolt7": 0.000000, "auxvolt6": 0.000000, "auxvolt5": 0.000000, "auxvolt4": 0.000000, "auxvolt3": 0.000000, "auxvolt2": 0.000000, "auxvolt1": 0.000000, "auxvolt0": 0.000000, "voltagevsen2": 0.000000, "voltagev5sb": 0.000000, "voltagevtt": 0.000000, "voltagevbat": 0.000000, "voltagev12p": 0.000000, "voltagev5p": 0.000000, "voltagev33stby": 0.000000, "voltagev33main": 0.000000, "voltagevcc1": 0.000000, "voltagevcc0": 0.000000, "numcpus": "1", "memusagepcnt": 21.040037, "memuseinmb": "215", "addimgmtcpuusagepcnt": 0.000000, "mgmtcpu0usagepcnt": 0.900000, "mgmtcpuusagepcnt": 0.900000, "pktcpuusagepcnt": 0.500000, "cpuusagepcnt": 0.500000, "rescpuusagepcnt": 4294967295.000000, "starttimelocal": "Thu Apr 21 05:11:10 2022", "starttime": "Thu Apr 21 05:11:10 2022", "disk0perusage": 22, "disk1perusage": 38, "cpufan0speed": 0, "cpufan1speed": 0, "systemfanspeed": 0, "fan0speed": 0, "fanspeed": 0, "cpu0temp": 0, "cpu1temp": 0, "internaltemp": 0, "powersupply1status": "NOT SUPPORTED", "powersupply2status": "NOT SUPPORTED", "powersupply3status": "NOT SUPPORTED", "powersupply4status": "NOT SUPPORTED", "disk0size": 1585, "disk0used": 325, "disk0avail": 1133, "disk1size": 14179, "disk1used": 5051, "disk1avail": 7993, "fan2speed": 0, "fan3speed": 0, "fan4speed": 0, "fan5speed": 0, "auxtemp0": 0, "auxtemp1": 0, "auxtemp2": 0, "auxtemp3": 0, "timesincestart": "00:00:00", "memsizemb": "0" } }root@control-plane:~/harora/cic/bugs/dns#

Here, "system.cpuusagepcnt" is 0.500 which is the same in CLI output.
which version of VPX are you using ?

[BraydenNeale]

I'm experiencing the same issue with the image: https://quay.io/repository/citrix/citrix-k8s-ingress-controller
version: quay.io/citrix/citrix-k8s-cpx-ingress:13.1-37.38
NS13.1 37.38.nc

CPU Usage through /nitro/v1/stat/system is always 4294967295

{
    "errorcode": 0,
    "message": "Done",
    "severity": "NONE",
    "system": {
        "voltagev12n": 0.000000,
        "voltagev5n": 0.000000,
        "cpuusage": "4294967295",
        "rescpuusage": "0",
        "slavecpuusage": "0",
        "mastercpuusage": "0",
        "auxvolt7": 0.000000,
        "auxvolt6": 0.000000,
        "auxvolt5": 0.000000,
        "auxvolt4": 0.000000,
        "auxvolt3": 0.000000,
        "auxvolt2": 0.000000,
        "auxvolt1": 0.000000,
        "auxvolt0": 0.000000,
        "voltagevsen2": 0.000000,
        "voltagev5sb": 0.000000,
        "voltagevtt": 0.000000,
        "voltagevbat": 0.000000,
        "voltagev12p": 0.000000,
        "voltagev5p": 0.000000,
        "voltagev33stby": 0.000000,
        "voltagev33main": 0.000000,
        "voltagevcc1": 0.000000,
        "voltagevcc0": 0.000000,
        "numcpus": "2",
        "memusagepcnt": 87.227531,
        "memuseinmb": "408",
        "addimgmtcpuusagepcnt": 0.000000,
        "mgmtcpu0usagepcnt": 0.000000,
        "mgmtcpuusagepcnt": 0.000000,
        "pktcpuusagepcnt": 0.300000,
        "cpuusagepcnt": 4294967295.000000,
        "rescpuusagepcnt": 4294967295.000000,
        "starttimelocal": "Tue Jul 25 07:29:02 2023",
        "starttime": "Tue Jul 25 07:29:02 2023",
        "disk0perusage": 0,
        "disk1perusage": 0,
        "cpufan0speed": 0,
        "cpufan1speed": 0,
        "systemfanspeed": 0,
        "fan0speed": 0,
        "fanspeed": 0,
        "cpu0temp": 0,
        "cpu1temp": 0,
        "internaltemp": 0,
        "powersupply1status": "NOT SUPPORTED",
        "powersupply2status": "NOT SUPPORTED",
        "powersupply3status": "NOT SUPPORTED",
        "powersupply4status": "NOT SUPPORTED",
        "disk0size": 0,
        "disk0used": 0,
        "disk0avail": 0,
        "disk1size": 0,
        "disk1used": 0,
        "disk1avail": 0,
        "fan2speed": 0,
        "fan3speed": 0,
        "fan4speed": 0,
        "fan5speed": 0,
        "auxtemp0": 0,
        "auxtemp1": 0,
        "auxtemp2": 0,
        "auxtemp3": 0,
        "timesincestart": "00:00:00",
        "memsizemb": "0"
    }
}

[subashd]

hi @BraydenNeale
We are looking into the issue.

[subashd]

hi @BraydenNeale
Some counters are outdated in the above output and we are planning to remove the same.
For CPU usage by mgmt and PE please use “mgmtcpuusagepcnt” and “pktcpuusagepcnt” counters.

[stevtampa]

I am having the same issues. Was this confirmed to be a bug with a certain nitro version and we need to upgrade? Can anyone advise?

[subashd]

@stevtampa as mentioned you can use “mgmtcpuusagepcnt” and “pktcpuusagepcnt” counters for CPU Usage by Mgmt and PE.
Which version of NetScaler are you using?