when setting the Accept-Encoding header to None, it is still sent as 'identity'

[vdanen]

I am passing these headers to request:

headers = {'Content-Type': 'application/something+xml',
'Accept': 'application/something+xml',
'Host': host,
'Expect': '100-continue',
'Connection': None,
'Accept-Encoding': None}

and then using:

r = requests.put(my_url, data=strata_xml, headers=headers, auth=(user, pwd), verify=False)

the REST service on the other end does not like this header at all. Using nc, I see it sending:

PUT /myurl HTTP/1.1
Accept-Encoding: identity
Content-Length: 1200
Accept: application/something+xml
User-Agent: python-requests/2.4.1 CPython/2.7.8 Darwin/13.3.0
Host: localhost
Expect: 100-continue
Content-Type: application/something+xml

I believe that https://hg.python.org/cpython/file/d047928ae3f6/Lib/http/client.py#l1086 may offer some clue there as to why this header is not being deleted when I set it to None, but I don't know how to work-around that.

Thanks.

[vdanen]

Essentially http.client adds "Accept-Encoding: identity" if there is no "Accept-Encoding" header present in the prepared request and this skip_accept_encoding option is not set (to clarify further)

[sigmavirus24]

@vdanen we remove None before passing the headers on to urllib3. urllib3 does not do the same processing that we do on headers before passing them on to httplib/http.client (as it shouldn't). There's no way to pass this on to urllib3 in such a way that it can also tell httplib that. Is there any Accept-Encoding value that this server won't balk at?

[vdanen]

Not that I have found, no. It doesn't like that header at all it seems. I've also tried to use a generator to force it to be chunked (which is the only way this header doesn't get sent it seems), but the server really didn't like that.

I've asked if there is a way they can fix their end to accept/discard/ignore that header without dying, but I probably won't hear back on that for a bit, and I may not be the only one to run into this problem which is I reported it.

[Lukasa]

That's really fairly terrible. Accept-Encoding: identity is always valid, the RFCs say so. It should be utterly harmless to send it along.

Otherwise, removing this requires us to replace httplib. That's a substantial bit of work. =(

[vdanen]

Yes, I see that. And I certainly wouldn't expect to have to gut this just to make it work for my use-case which seems to not be that standard. Thanks for the info! If it's not possible to work-around this in any reasonable way, then there probably isn't a point in keeping this open.

[oxbambooxo]

😢

[forgetso]

Need the same thing for Connection:keep-alive - don't send it. :-(

[sigmavirus24]

This is an issue from 2014 that is not related to the current comments. To avoid reviving a long dead issue, I'm locking the conversation.

@forgetso because your comment has actionable information, I'll inform you that Connection: keep-alive is the default for HTTP/1.1. If you don't want keep-alive behaviour, you can specify Connection: close instead.