nginx_connections_unhandled_total appears incorrect

[misterbisson]

nginx_connections_unhandled_total is being reported by the ContainerPilot telemetry sensor script per the config. The number is expected to be zero most of the time, but Prometheus shows negative numbers:

[misterbisson]

After thinking on this a moment, the problem seems obvious:

unhandled() {
    local accepts=$(curl -s --fail localhost/nginx-health | awk 'FNR == 3 {print $1}')
    local handled=$(curl -s --fail localhost/nginx-health | awk 'FNR == 3 {print $2}')
    echo $(expr ${accepts} - ${handled})
}

The connections that have been handled is higher than those it has accepted because the two are being determined from separate requests to localhost/nginx-health. Duh.

We do the correct thing right below that in the same file:

connections_load() {
    local scraped=$(curl -s --fail localhost/nginx-health)
    local active=$(echo ${scraped} | awk '/Active connections/{print $3}')
    local waiting=$(echo ${scraped} | awk '/Reading/{print $6}')
    local workers=$(echo $(cat /etc/nginx/nginx.conf | perl -n -e'/worker_connections *(\d+)/ && print $1')
)
    echo $(echo "scale=4; (${active} - ${waiting}) / ${workers}" | bc)
}

Note that it captures the values from localhost/nginx-health in a single request, then parses them out as needed, rather than making multiple requests.