Support empty HTTP header values

[pschorf]

As I understand it, RFC 7230 supports headers with empty values:

field-value = *( field-content / obs-fold )

But HttpGenerator has the comment

// rfc7230 does not allow no value

on line 605 which skips headers with no value. Am I misreading the spec? If not, would it be possible to update Jetty to support passing these headers?

[joakime]

An empty header field-value is not valid per RFC7230.

https://tools.ietf.org/html/rfc7230#section-3.2

The field-value is not optional and must be specified.

There is only 1 exception made for this rule, and it only applies to the Client and its declaration of the "Host" request header under some very specific conditions.

All server generated headers are never empty, as this is cause of many security issues (mostly with intermediaries).

Have you tried just sending an empty quoted string? ""

[pschorf]

I did some more research and after reading the errata I agree. I maintain a proxy so I'll need to investigate whether or not any clients actually use this feature but it came up as a regression.

Thanks for taking the time to reply.

[jamshid]

An empty header field-value is not valid per RFC7230.

According to the people that should know, that is not correct. Empty header values are valid http, see mnot's reply in whatwg/fetch#332.

Breaking your users on upgrade is never a good idea. Please fix jetty to continue supporting empty headers.

[joakime]

@gregw you'll want to review this.

[joakime]

@jamshid btw, if you want RFC2616 behavior (where empty header values were allowed), you can configure your Jetty Connector to use RFC2616 behavior. But be aware that using empty headers with modern browsers can result in unexpected behavior, as they have universally adopted the updated RFCs (this isn't limited to RFC2616 vs RFC7230 btw, but all associated RFCs).

[gregw]

Having read RFC7230 again, specifically:

     field-value    = *( field-content / obs-fold )
     field-content  = field-vchar [ 1*( SP / HTAB ) field-vchar ]

That's clear that an empty field-value is acceptable due to the *() rule. So I'm wondering exactly where we got the idea that it was not? I'm guessing at this stage it was just a miss-reading of the ABNF, which is a bit convoluted.... in fact it is a real mess... field-content can be a single character OR a single character followed by 1 or more space/tabs followed by another single character.

So a value like pigs can \t fly breaks down as:

• field-content p
• field-content i
• field-content g
• field-content s c
• field-content a
• field-content n \t f
• field-content l
• field-content y

No wonder we may have been confused!

I'll prepare a fix now, but will do a bit more research as well to truly understand if we did this for a reason or were just confused by horrid ABNF?

[gregw]

So HttpParser was not enforcing no empty headers, it only enforces a missing colon.
It was only HttpGenerator that was skipping headers with null or empty values.

It is a trivial fix to make the generator allow empty values, but currently I have it still skipping null values (because adding a null value to our HttpFields class is the same as removing that entry)

Note that the jetty compliance mode will not help in the HttpGenerator, as that appears to have been added without a check of the compliance mode anyway! Nor was there a unit test for that behaviour?!?! It was my commit - so my bad !!

I'll push the simple fix plus test to check it is fixed soon, but I think we need to check higher layers as well to make sure that empty and/or null values are being correctly handled.

@jamshid @pschorf do you have a particular use-case we can verify?