dart:io client WebSocket can not connect to some WebSocket servers

[vadimtsushko]

Some problem description is at https://groups.google.com/a/dartlang.org/forum/#!topic/misc/hPBiK2OlLs0

Specifically I have a trouble connection to Qlik Sense Engine API WebSocket service.
I believe problem is Qlik Sense expects handshake header keys in capitalized form and
dart:io HttpHeaders keys are always converted to lowercase.

[whesse]

@sgjesse @mkustermann
Here is the relevant information from the email thread: The problem seems to be that dart:io lowercases header names, and Qlik web socket code only accepts mixed case header names:

Vadim said:
So I spent some time with WireShark to find what is different in WebSocket handshaking of nodejs (successful) and dart:io (unsuccessful) communication. For now I'm almost totally sure that difference and reason why dart:io WebSocket connections break with error is that dart:io header keys are converts to lowercase.

Successful communication looks like:

Here is the relevant information from the email thread: The problem seems to be that dart:io lowercases header names, and Qlik web socket code only accepts mixed case header names:

Vadim said:
So I spent some time with WireShark to find what is different in WebSocket handshaking of nodejs (successful) and dart:io (unsuccessful) communication. For now I'm almost totally sure that difference and reason why dart:io WebSocket connections break with error is that dart:io header keys are converts to lowercase.

Successful communication looks like:

GET /app/%3Ftransient%3D HTTP/1.1
Connection: Upgrade
Upgrade: websocket
Host: 192.168.188.10
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: MTMtMTQ0OTMzNjYzNzY0NQ==
Content-Type: application/json
Cookie: X-Qlik-Session=70a54c5b-a08b-4665-a0d7-1a39871ebea9; Path=/; HttpOnly; Secure
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: Tb5mxSaTiFApEyYAAv/ci7AMOzI=
Access-Control-Allow-Origin: ws://192.168.188.10

Unsuccessful communication looks like:

GET /app/%3Ftransient%3D HTTP/1.1
user-agent: Dart/1.14 (dart:io)
connection: Upgrade
cache-control: no-cache
accept-encoding: gzip
cookie: X-Qlik-Session=70a54c5b-a08b-4665-a0d7-1a39871ebea9; Path=/; HttpOnly; Secure
content-length: 0
sec-websocket-version: 13
host: 192.168.188.10
sec-websocket-extensions: permessage-deflate; client_max_window_bits
content-type: application/json
sec-websocket-key: lVH5YxHL8QjS7ddjIJ/Ohg==
upgrade: websocket

HTTP/1.1 404 Not Found
Content-Length: 0
Access-Control-Allow-Origin: ws://192.168.188.10

I did not find any way to pass intact (not converted to lowercase) headers to dart:io WebSocket or HttpClient. So finally I've tried raw Socket and with it (writing to it capitalized Connection and Upgrade as header keys) I've managed to get switching protocol response from the Qlik Sense WebSocket server. So apparently from that I can proceed to create WebSocket from manually upgraded WebSocket.

But I wonder now- is Qlik Sense WebSocket server implementation is wrong to reject dart:io client WebSocket connections? Or dart:io WebSockets are somehow not strictly adhere to specification?
Dart:io WebSocket client successfully connects to http://www.websocket.org/ sample server, to dart:io WebSocket servers, to nodejs ws server - but how many Qlik Sense like not compatible with dart:io WebSocket servers are in the wild?

Communication example on Wikipedia looks more like nodejs version (with capitalized keys)

GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13
Origin: http://example.com


GET /app/%3Ftransient%3D HTTP/1.1
Connection: Upgrade
Upgrade: websocket
Host: 192.168.188.10
Sec-WebSocket-Version: 13
Sec-WebSocket-Key: MTMtMTQ0OTMzNjYzNzY0NQ==
Content-Type: application/json
Cookie: X-Qlik-Session=70a54c5b-a08b-4665-a0d7-1a39871ebea9; Path=/; HttpOnly; Secure
Sec-WebSocket-Extensions: permessage-deflate; client_max_window_bits

HTTP/1.1 101 Switching Protocols
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Accept: Tb5mxSaTiFApEyYAAv/ci7AMOzI=
Access-Control-Allow-Origin: ws://192.168.188.10

Unsuccessful communication looks like:

GET /app/%3Ftransient%3D HTTP/1.1
user-agent: Dart/1.14 (dart:io)
connection: Upgrade
cache-control: no-cache
accept-encoding: gzip
cookie: X-Qlik-Session=70a54c5b-a08b-4665-a0d7-1a39871ebea9; Path=/; HttpOnly; Secure
content-length: 0
sec-websocket-version: 13
host: 192.168.188.10
sec-websocket-extensions: permessage-deflate; client_max_window_bits
content-type: application/json
sec-websocket-key: lVH5YxHL8QjS7ddjIJ/Ohg==
upgrade: websocket

HTTP/1.1 404 Not Found
Content-Length: 0
Access-Control-Allow-Origin: ws://192.168.188.10

I did not find any way to pass intact (not converted to lowercase) headers to dart:io WebSocket or HttpClient. So finally I've tried raw Socket and with it (writing to it capitalized Connection and Upgrade as header keys) I've managed to get switching protocol response from the Qlik Sense WebSocket server. So apparently from that I can proceed to create WebSocket from manually upgraded WebSocket.

But I wonder now- is Qlik Sense WebSocket server implementation is wrong to reject dart:io client WebSocket connections? Or dart:io WebSockets are somehow not strictly adhere to specification?
Dart:io WebSocket client successfully connects to http://www.websocket.org/ sample server, to dart:io WebSocket servers, to nodejs ws server - but how many Qlik Sense like not compatible with dart:io WebSocket servers are in the wild?

Communication example on Wikipedia looks more like nodejs version (with capitalized keys)

GET /chat HTTP/1.1
Host: server.example.com
Upgrade: websocket
Connection: Upgrade
Sec-WebSocket-Key: x3JJHMbDL1EzLkh9GBhXDw==
Sec-WebSocket-Protocol: chat, superchat
Sec-WebSocket-Version: 13
Origin: http://example.com

[sgjesse]

Thanks for the report, however the current behaviour is by design.

The HTTP/1.1 specification section 4.2 clearly states that the field names in HTTP headers are case-insensitive. Early on in the design of the Dart HTTP library we decided to make all header fields lowercase, and not try to neither keep the passed-in case nor produce the Xxx-Yyyy-Zzz format.

Btw. the HTTP/2 specification went with all lowercase headers.

[mike-mathieu]

Has anyone found a workaround for this on the client side as I've ran into the same issue?

[zoechi]

@bigmikehoncho I think you should try to get the server fixed. Have you tried to report it as a bug?

[mike-mathieu]

thanks @zoechi they were able to change their protocol.

[Murtuzakabul]

Is there a way in dart to override this default behavior. It is possible to pass the headers but they too are getting converted to lowercase.