Support HEAD requests

[vongruenigen]

We witness a strange behaviour of mod_zip when different HTTP methods are used.

If I make a GET request to our application, it correctly generates the file list and sends it back to nginx, which then generates the ZIP file and sends it back to the client. Everything working as expected and nginx does not log any error or similar.

Now, I'll do the same request, but this time as a HEAD request. The request hits our application, it generates the exact same file list and sends it back to nginx. But this time nginx logs the following error:

2013/12/12 08:21:49 [error] 25049#0: *45414 mod_zip: invalid file list from upstream while sending to client, client: 192.168.1.10, server: _, request: "HEAD /abc/download HTTP/1.1", upstream: "http://unix:/var/run/myapp/server.socket:/abc/download", host: "myapp.com"

The interessting thing is that our application is not able to differ between HEAD and GET requests. So the HEAD request is handled exactly the same way as the (working) GET request is.

[vongruenigen]

@evanmiller, any idea what the problem could be?

[pgaertig]

@vongruenigen, IMO it is not related with mod_zip at all. The module it is exactly pointing the cause of the problem - no content from upstream. If your application server is HTTP 1.1 standard compliant it may run some body generating content but for sure it won't sent any content back to nginx.

http://www.w3.org/Protocols/rfc2616/rfc2616-sec9.html#sec9.4
"The HEAD method is identical to GET except that the server MUST NOT return a message-body in the response.".

I suppose the solution is to convert any upstream HEAD requests to GET.
You can try proxy_method directive:

proxy_method GET;

If that helps you please share the result here. I need to update my configs to support HEAD soon as well.

[evanmiller]

@pgaertig, thanks for the detective work. It would be nice if mod_zip supported HEAD requests out of the box. This is not something I plan to work on but patches welcome.

[pgaertig]

I have tried "proxy_method GET;" and it is bad. Nginx is responding with ZIP body for HEAD :/

[pgaertig]

I have put this quickly

if (r->method == NGX_HTTP_HEAD) {
  /* Don't send any content if HEAD request method was used */
  return ngx_http_send_special(r, NGX_HTTP_LAST);
} else {
  return ngx_http_zip_send_pieces(r, ctx); 
}

instead of line https://github.com/evanmiller/mod_zip/blob/master/ngx_http_zip_module.c#L543
However it doesn't work for me, the content is still downloaded. I suppose my customized nginx source deb package may be screwed. Could somebody check?

Below is an example curl to play with it. It should return expected Content-Length but the reported body download should be zero bytes.

curl -o /dev/null -XHEAD -v  http://theserver/path_to.zip

[vongruenigen]

@pgaertig, thanks for investigating. I'll try your patch as soon as I've some spare time.

[dup2]

It looks IMHO like the HEAD request to the backend (upstream) returns no content but a header X-Archive-Files which triggers mod_zip to parse the content which in turn fails as it does not contain a valid file list.

Line 543 might be too late to catch this - the response body is already parsed on line 511 and mod_zip will throw an error. We might check for HEAD requests early at https://github.com/evanmiller/mod_zip/blob/master/ngx_http_zip_module.c#L484 and return there.

[pgaertig]

My patch attempt should be really used with proxy_method GET to get list from upstream with GET, then mod_zip can calculate ZIP size and return it in Content-Length whatever HTTP method was used. In case of HEAD like the standard advises I want nginx to respond with everything which is present in GET response, especially headers, except body.

If you want to not return any meaningful data and headers then 405 Method Not Allowed with Allow: GET header should be returned I suppose at line you pointed.

I am almost sure HEAD is used by JDownloader and other downloading software to check file availability and properties without actually downloading it.

[dup2]

Good point - our client is actually a iOS newsstand application which checks the URL with a HEAD before downloading.

[Artlu]

I'm also affected by this error.

I'm generating the responses to enable mod_zip with a php file.
As a workaround, I am not sending the "X-Archive-Files: zip"-header when there's a HEAD request. Also no Content-Length gets sent and no body, but the "Content-Disposition"-header so applications can get the correct filename.

As far as I can tell, most applications which do a HEAD request at start can handle this behaviour just fine, but that's not a solution which I would accept permanently.

[dcsobral]

And here I was thinking this was about mod_zip sending HEAD requests to get the size of every file...