net/http: Content-Length: 0 is not set for PATCH requests with empty body

[tharinduwijewardane]

What version of Go are you using (go version)?

1.15

Does this issue reproduce with the latest release?

yes

What operating system and processor architecture are you using (go env)?

darwin amd64

go env Output

GO111MODULE=""
GOARCH="amd64"
GOBIN="/Users/tharindu/GoCode/bin"
GOCACHE="/Users/tharindu/Library/Caches/go-build"
GOENV="/Users/tharindu/Library/Application Support/go/env"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="darwin"
GOINSECURE=""
GOMODCACHE="/Users/tharindu/GoCode/pkg/mod"
GONOPROXY=""
GONOSUMDB=""
GOOS="darwin"
GOPATH="/Users/tharindu/GoCode"
GOPRIVATE=""
GOPROXY="https://proxy.golang.org,direct"
GOROOT="/usr/local/Cellar/go/1.15/libexec"
GOSUMDB="sum.golang.org"
GOTMPDIR=""
GOTOOLDIR="/usr/local/Cellar/go/1.15/libexec/pkg/tool/darwin_amd64"
GCCGO="gccgo"
AR="ar"
CC="clang"
CXX="clang++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fno-caret-diagnostics -Qunused-arguments -fmessage-length=0 -fdebug-prefix-map=/var/folders/3l/cw1f6lvn6p1c_vcg0cnbmqwc0000gp/T/go-build547496291=/tmp/go-build -gno-record-gcc-switches -fno-common"

What did you do?

package main

import (
	"fmt"
	"io/ioutil"
	"net/http"
	"strings"
)

func main() {

	url := "https://myapi.net/logs/file/a?action=aa"
	method := "PATCH"

	payload := strings.NewReader("")
	client := &http.Client {
	}
	req, err := http.NewRequest(method, url, payload)

	if err != nil {
		fmt.Println(err)
	}
	req.Header.Set("Authorization", "Bearer my-token")
	req.Header.Set("Content-Length", "0")

	res, err := client.Do(req)
	defer res.Body.Close()
	body, err := ioutil.ReadAll(res.Body)

	fmt.Println(string(body))
}

What did you expect to see?

Content-Length header getting set to 0 when the request is observed from the server side.

What did you see instead?

Content-Length header is not getting set for PATCH requests with empty/nil payload. It is getting set without any issue for other http methods (POST, PUT).
It is also getting set for PATCH requests if the payload is non empty.
Also setting req.Header.Set("Content-Length", "0") seems to have no effect on the actual content length getting set in the request.

[odeke-em]

Thank you for filing this issue @tharinduwijewardane and welcome to the Go project!

I’d like to understand the motivation here. Was there a failure on some intermediate or front end server, say Nginx or is your service breaking due to lack of that 0? I ask because in 9+ years no one had reported this being a problem. Did you discover this perhaps just by a code audit, or an actual failure? Please let us know. The impetus for adding that 0 for POST and PUT is that RFC 2616 says we can include 0 only if the method doesn’t prohibit it, and Nginx apparently needs that too. However, nothing has been reported/requested for DELETE nor PATCH in the past 9 years (almost decade).

[tharinduwijewardane]

@odeke-em I came across this when invoking the Azure data lake storage gen2 API (https://docs.microsoft.com/en-us/rest/api/storageservices/datalakestoragegen2/path/update) for flushing the content of a file which was uploaded in a previous request.
They deliberately reject the PATCH request when the Content-Length is not set to 0 even if there is no content. Maybe that is something they should not do.

However, at-least we should be able to manually set the content-length header right?
In this case req.Header.Set("Content-Length", "0") is not honoured by http package.
I think we should be able set whatever the header we need.

[odeke-em]

Thank you for providing the background, @tharinduwijewardane! I have cited your case in the fixing CL https://go-review.googlesource.com/c/go/+/250039. To get around this, firstly am away from my computer for a couple of hours but I’ve written for you a repro with a stand-alone server using my iPad here https://play.golang.org/p/43_KR2riBiT, please let me know what it prints out on your computer.

[gopherbot]

Change https://golang.org/cl/250039 mentions this issue: net/http: set Content-Length:0 for empty PATCH requests as with POST, PATCH

[tharinduwijewardane]

@odeke-em "In server" the content-length header is not received.

PreRequest PATCH / HTTP/1.1
Host: 127.0.0.1:65209
Content-Length: 0


In server PATCH / HTTP/1.1
Host: 127.0.0.1:65209
Accept-Encoding: gzip
User-Agent: Go-http-client/1.1


Response HTTP/1.1 200 OK
Date: Mon, 24 Aug 2020 08:17:40 GMT
Content-Length: 0

[odeke-em]

Thank you for running them! Given that you are the sole reporter in the past 9 years, I don’t think that this could qualify for a backport, but would it be alright for you to use go tip for the next 3+ months until Go1.16 is released? If not, let me know and I can craft up for you a way of hacking in that Content-Length:0 or perhaps request backports to the last 2 Go releases

[davecheney]

Are you sure that rfc2616 defines patch? From my research it was proposed in 2010 in https://tools.ietf.org/html/rfc5789

[odeke-em]

It just says that as long as the method doesn’t forbid sending the Content-Length, it can be sent which is my interpretation to say that it doesn’t forbid it.

…

On Mon, Aug 24, 2020 at 1:28 AM Dave Cheney ***@***.***> wrote: Are you sure that rfc2616 defines patch? From my research it was proposed in 2010 in https://tools.ietf.org/html/rfc5789 — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40978 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFL3V4B3A7EMLKVNIAVOOTSCIQC5ANCNFSM4QIEEM2Q> .

[tharinduwijewardane]

@odeke-em Thank you. But this is no longer a blocker for me as there is a workaround/hack.
By setting req.TransferEncoding = []string{"identity"} I can get it to add the header.

[davecheney]

It just says that as long as the method doesn’t forbid sending the Content-Length, it can be sent which is my interpretation to say that it doesn’t forbid it.
…

That’s probably for connection reuse to work.

What’s the use case for sending a PATCH with an empty body, surely the server will reject it?

[odeke-em]

Thanks Dave for the citations. In regards to the use case, the original reporter stated that they need to send an empty PATCH request in order to flush the contents of a file that was previously uploaded, in reply to my similar question as per #40978 (comment)

…

On Mon, Aug 24, 2020 at 1:36 AM Dave Cheney ***@***.***> wrote: It just says that as long as the method doesn’t forbid sending the Content-Length, it can be sent which is my interpretation to say that it doesn’t forbid it. … <#m_-5160185177206370588_> That’s probably for connection reuse to work. What’s the use case for sending a PATCH with an empty body, surely the server will reject it? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#40978 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABFL3V443H3YAWDWSSQSOS3SCIRCHANCNFSM4QIEEM2Q> .

[davecheney]

Ok, thanks for confirming