Cookie name/value serialization

[annevk]

Since cookie values at least are a byte sequence, it needs to be specified how these are converted to a string and vice versa.

[jgraham]

Oh no, I guess we don't handle this at all. It would be really useful to know what happens in implementations today if you make a cookie with a non-UTF8 serializable value.

[annevk]

That only works in Chrome at the moment, but we are considering aligning Firefox with that behavior (Fx is limited to UTF-8). Safari only supports ASCII I think, but I did not test exhaustively. (Those cookies in Chrome are not serialized for document.cookie.)

[johannhof]

From a quick conversation with @annevk we think it makes sense to transfer raw bytes from the browser without parsing into UTF-8 or so in between. To do that we could pass a number array or a ByteString-like format.

[sbingler]

Perhaps an uninformed take (please correct me if so) but from what I understand one of the sticking points is trying to pick a useful representation of the cookie's octets.

I like the idea of a byte string, and it may in fact be necessary to support non-printable cookie octets, but that's not useful to the human on the other end. What if, in addition to the byte field, there was also a human readable field? Any non-printable characters would be given some default printable value but it would otherwise represent the same data.

This would allow a testdriver.js user to more easily compare the results of their query to their expectation. Instead of having them manually convert foo=bar to 0x660x6F0x6F=0x620x610x72. They could just look at the name_human_readable and value_human_readable(name tbd) fields instead.

If the expected cookie has non-printable characters in it then you'd of course have to look at the byte string field instead. But considering that the vast majority of our cookie tests use printable characters that seems fine.

I suppose you could also just add a helper function that the user could call to do the human readable conversion, but having it built into the JSON object seems appealing.

[jgraham]

From a spec point of view the main concern I have is compatibility; we can't change the existing fields from a string to e.g. an array since that will break consumers. We probably can change it from a plain string to an encoded string as long as the encoding is a noop for UTF8 values. We could add an extra field but we'd still need to specify how to serialize the existing field.

[annevk]

I'm confused by the example above. A ByteString (see Web IDL) is ASCII-compatible so "foo=bar" would be "foo=bar". In fact, all bytes get mapped to their code point of equal value. This mainly looks weird if those bytes also happen to be UTF-8-compatible byte sequences. As this is mainly for testing I'm not sure it's worth optimizing for that, but I suppose one could invent a format that mostly looks like normal Unicode strings but can also represent arbitrary bytes through some kind of escape hatch?

[sbingler]

Seems like it was an uninformed take, thanks for your feedback!

As I understand it the sticking point is that bytestrings cannot be directly represented within JSON. I assume this is at least partially due to the fact that some bytes can map to non-printable characters while JSON is meant to be human readable.
(The unicode chart for basic latin seems to imply that these should all have printable representations, but my attempts to copy paste here were not fruitful)

Is there any reason not to convert to a JSON string, including escaping non-printable characters, directly? As you mentioned, Anne, this is primarily for testing so for the majority of cases it should "just work" while users testing for control characters or the like would just need to expect escape sequences in the returned fields.

[annevk]

ByteString can only contain U+0000 through U+00FF (each representing a byte). While not all of those are "printable", I don't think JSON actually puts limits on those particular code points?

[jgraham]

Oh I see the proposal is to use a string, as the JSON side encoding, but to hold the byte representation directly. If Gecko and Blink are already allowing UTF-8 strings that feels like it could break existing users. It also seems pretty surprising; pretty much any modern consumer is going to convert a JSON string into the language's Text type by default, and it's very unusual for that to actually contain bytes.

I think I'd prefer the case where we know we have valid UTF8 bytes to just encode them as a string directly in the value field. Then in the case where the bytes can't be decoded as UTF8 we also have a value_bytes field that's an array of integers. In this case I'm not sure what should happen with the value field (e.g. whether it should be some attempt to partially encode the value as a string, or if it should be empty, or null or something else. In any case it seems like an edge case that an insufficiently careful client is unlikely to handle well),

[annevk]

For ease of usage I'd probably always include both fields then. Seems a bit annoying for consumers to have to branch.