Pushing empty label values returns 405 Method Not Allowed

[matevz]

Using client_golang:

myPusher = myPusher.Grouping("firstlabel", "abc")
myPusher = myPusher.Grouping("secondlabel", "")
myPusher = myPusher.Grouping("thirdlabel", "def")

results in generating PUT HTTP request

http://mypushgateway:9091/metrics/job/myjob/firstlabel/abc/secondlabel//thirdlabel/def

and when sending it to push gateway, it responds 405 Method Not Allowed.

I'm not sure whether that's a push gateway caveat, URL encoding problem or client_golang issue, but empty label values are perfectly legal in Prometheus and should work fine.

Using pushgateway 1.2.0 and client_golang 1.5.1.

[beorn7]

I guess it's mostly an URL encoding problem. Will investigate…

[beorn7]

Sooo, this is even a bit deeper:

If a double-slash (//) is encountered, the Go ServeMux creates an HTTP 301 redirect to the cleaned path. Proper HTTP clients then change the method to GET when they encounter a redirect because POST and PUT are not considered idempotent. That's where your 405 is coming from. We could fix that change of method by sending a 307 or 308, but that wouldn't help us here because then the Pushgateway will complain about the wrong number of components.

We could probably hack deeply into the HTTP routing code to somehow allow // to make it through without a redirect, but that will be tedious and dirty.

The use case for empty grouping label values is probably rather narrow, but as you say, it's formally valid and I could even dream up some scenarios where it makes a difference from just leaving out the label with the empty value altogether (same reason why the PGW adds an empty instance label by default).

Luckily, there is a work-around using base64 encoding. Your PUT works as intended with the following URL: http://mypushgateway:9091/metrics/job/myjob/firstlabel/abc/secondlabel@base64/=/thirdlabel/def

Now we only need to teach that to the client libraries…

I'll send a PR to update the docs here on the PGW side.

[beorn7]

With the updated docs, this issue is handled from the PGW side. Fixes for client libraries are happening as I'm typing this.