Extension names become case insensitive in binary encoding

[inlined]

Extensions are not defined to have case-insensitive names, but the binary encoding puts the extension name in a header. Since header names are case insensitive, their case may be changed by middleware. This makes extensions fairly fragile.

I suggest we either:

1. Define extensions to be case insensitive in spec.md
2. Change the way extensions are encoded in HTTP binary encoding.

I personally prefer (2). I'd recommend

CE-X: <key>: <value>

[tjanczuk]

+1.

This problem makes it impossible to implement a compliant consumer in some popular frameworks. For example, the Node.js HTTP implementation explicitly normalizes all HTTP headers to lowercase.

[duglin]

I'd prefer a solution that didn't treat extensions any differently from spec-approved attributes. Is there an issue with having the same rules for both?

[inlined]

We could just have a single header CE-Extensions that is a map of all extensions? I think labels are also something that we might want to put in headers always and use the same sort of repeated header rule for easy filtering by gateways.

[duglin]

A single header would be ok as long as the extensions are actually modeled as sub-properties of something. E.g. if in JSON they were:

"extensions": {
  "foo": "bar"
}

If we allowed extensions to be top-level json properties then they're more akin to siblings of spec-approved attributes and I would prefer for them to be individual http headers.

Yes, I'm still hoping to convince people we don't need a formal "bag" defined and we can just use normal json (or whatever) extension mechanisms. :-)

[tjanczuk]

Many HTTP stacks impose limits on individual header size, which may make a single CE-Extensions header less practical than a header per attribute approach. Also, assuming the intent of putting extensions in headers in the first place is to allow for easy processing by intermediaries, I like the CE-X: <key>: <value> @inlined proposed above best.

[inlined]

As my own devil's advocate, some HTTP libraries will get in the way of parsing this. HTTP requires the result of

HeaderFoo: Bar
HeaderFoo: Baz

to be the same as

HeaderFoo: Bar,Baz

Since lists of JSON objects are not easily parsed with commas, this could make parsing harder. I almost wish the keys were themselves quoted so you could just hack it by wrapping the value in [] before tossing it into an off-the-shelf parser.

[technosophos]

Encoding the header values with base64 will allow easy parsing

In pseudo-code:

CE-X: BASE64(name: value)
CE-X: BASE64(name2:value2)

Which then easily meets the HTTP spec's header collapsing rules as:

CE-X: BASE64(name: value), BASE64(name2: value2)

Other options include:

• use multi-part MIME bodies and store extensions in one part
• Using query strings (which has some interesting consequences)
• defining an encoding format for extensions that are embedded in headers
• requiring strings to be map[string]string instead of map[string]any

[duglin]

See: #225

[duglin]

This topic came up on the SDK call yesterday. I'm wondering if we need to do anything about this at all from a spec perspective. Meaning, we're just following HTTP and treating the headers as case insensitive. How they get mapped in the tooling is up to the tooling. (punt!)

So , in the case of extensions, I would think that in most cases the tooling would just put them into some kind of map. Is the solution here anything more complicated than the tooling (sdk) be clear about what it does with them? For example, perhaps we decide that our SDK lower-cases all extension names so they're easier to find? It could them then "as-is" but I think that's a bit harder for app to deal with.

[timbray]

Does the spec say (don't recall seeing this) that properties (built-in or extensions) be English words? If not, the base64 idea is worth serious consideration, specifically for HTTP headers, which are a lossy encoding. Case-folding is a can of worms, and should be avoided wherever possible in the interests of sanity and diversity. E.g., if ASCII case-folding rules are applied mechanically, they are apt to break anyone who uses a Turkish word to name a property. Case has no place in Internet protocols.

[duglin]

One of the reasons we created the raw/binary format was because we wanted a format that avoided the JSON parser if all you wanted was the CE properties - for faster parsing times. If we base64 encode the properties it seems we're losing that benefit.

I still don't see the issue with the spec being agnostic about this - just let the protocol spec (e.g. http) define how properties are serialized, and separately let the SDKs decide how they will present extensions to their users. If the protocol is case in-sensitive then I think the SDK should lowercase them - but that a decision it can make for its users.