Skip to content

Add MessagePack support for Java SignalR Client #23532

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 64 commits into from
Aug 20, 2020
Merged

Conversation

wtgodbe
Copy link
Member

@wtgodbe wtgodbe commented Jun 30, 2020

Resolves #21644

Things to call out in docs:

  1. Users should pass in a Type when calling HubConnection.on(), HubConnection.invoke(), or HubConnection.stream(). You can get a Type w/ something like new TypeReference<ArrayList<Integer>>() { }).getType(). The exception is primitives, for which you should use, e.g., int.class. The methods mentioned are generic, but have no generic parameter, so must be invoked with the syntax HubConnection.<ArrayList<Integer>>invoke(Type, ...)
  2. Chars will be serialized as Strings. Chars are not defined by the messagePack spec, so its up to the library implementer to decide how to serialize them. The java library we use serializes them as Strings, which is different than the C# library we use, which serializes them as Shorts.

Todo:

  1. Separate msgPack into its own package
  2. Decide if we're making TypeReference public

@ghost ghost added the area-signalr Includes: SignalR clients and servers label Jun 30, 2020

private Object readValue(MessageUnpacker unpacker, Class<?> itemType) throws IOException {
// This shouldn't ever get called with itemType == null, but we return anyways to avoid NullPointerExceptions
if (itemType == null) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should I even account for itemType being null? In theory it should never happen (would indicate that a function took an argument of type null, or returned something of type null, which shouldn't be possible) but the test InvocationBinder returns null:

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could this happen given a protocol violation by the server?

If so, we should at the very least log an error and fail the corresponding client-to-server invocation or skip the corresponding server-to-client invocation. Closing the entire HubConnection with an error might be even better.

If this actually cannot happen short of memory corruption or similar, just assert.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It could happen if the server provides an InvocationBinder that returns null from getReturnType(invocationId) when sending a StreamItem (or if the client does the same when sending an invocation)

int length;
Object item = null;

switch(valueType) {
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't account for the MessagePack data being malformed - if the user passes a Map when the function expects a String, this should just throw a MessagePackException. The C# client appears to do the same

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does this surface to the HubConnection user?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A call to ParseMessages would throw a RuntimeException w/ "Error reading messagepack data" - JSONHubProtocol does the same:

} catch (IOException ex) {
throw new RuntimeException("Error reading JSON.", ex);
}

While the C# Client's method is TryParseMessages, so it just returns false if it runs into issues.

@davidfowl
Copy link
Member

Should we change the interface HubProtocol to deal in byte[] rather than Strings? As it is, I have to convert between the two using StandardCharsets.ISO_8859_1, otherwise bytes like 0x90 and 0x93 will get translated incorrectly when going from byte[] -> String -> byte[]. Changing the interface would require changes to JsonHubProtocol, WebSocketTransport, and DefaultHttpClient.

Yes, otherwise it kinda defeats the purpose.

@Pilchie
Copy link
Member

Pilchie commented Jul 22, 2020

👀

@wtgodbe wtgodbe force-pushed the wtgodbe/HubProtocol branch 2 times, most recently from 7073c17 to 90926f5 Compare July 30, 2020 20:35
@wtgodbe
Copy link
Member Author

wtgodbe commented Jul 30, 2020

From today's meeting:

Next step is to convert the abstraction to use ByteBuffer instead of String (all the way down to

@Override
public Completable send(String message) {
websocketClient.send(message);
return Completable.complete();
}
). This should be fairly mechanical. Reason being that OkHTTP converts strings to UTF8 on Send, which will break MessagePack, so we need to invoke its ByteString method instead.

@wtgodbe wtgodbe force-pushed the wtgodbe/HubProtocol branch from 6b4388a to 15df263 Compare July 30, 2020 21:58
@wtgodbe
Copy link
Member Author

wtgodbe commented Jul 30, 2020

Current state - I've converted everything to use ByteBuffer. Json is still working, but MsgPack fails immediately on parsing the first message - on inspection, the length header looks malformed. The payload is 36 bytes - first byte 0x1c (28), second byte 0xc2 (false), 3rd byte 0x96 (6-length array - this appears to be the first byte of the message). Will look into what's happening when I get back to this next week

@@ -136,7 +136,7 @@ Transport getTransport() {
}

this.baseUrl = url;
this.protocol = new JsonHubProtocol();
this.protocol = new MessagePackHubProtocol();
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Change this

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@BrennanConroy @halter73 should this just be a param to the HubConnection constructor?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We'll want a method on the builder similar to

public withHubProtocol(protocol: IHubProtocol): HubConnectionBuilder {

and yes, the protocol will be passed to the constructor.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Need to default it to JsonHubProtocol :D

@wtgodbe
Copy link
Member Author

wtgodbe commented Aug 4, 2020

Current issue: The ByteBuffer received here is ReadOnly:

Figuring out why

@dotnet dotnet deleted a comment from azure-pipelines bot Aug 5, 2020
@dotnet dotnet deleted a comment from azure-pipelines bot Aug 5, 2020
String handshakeResponseString = payload.substring(0, handshakeLength - 1);
// The handshake will always be a UTF8 Json string, so we can convert the ByteBuffer to that to read the beginning of the payload
String payloadStr = new String(payload.array(), StandardCharsets.UTF_8);
int handshakeLength = payloadStr.indexOf(RECORD_SEPARATOR) + 1;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we find the index from the original payload first, then make a string once with the known size?

@wtgodbe
Copy link
Member Author

wtgodbe commented Aug 6, 2020

@BrennanConroy 35f2653

@wtgodbe
Copy link
Member Author

wtgodbe commented Aug 6, 2020

/azp run

@wtgodbe wtgodbe force-pushed the wtgodbe/HubProtocol branch from d244b67 to 5a2357e Compare August 19, 2020 17:58
@wtgodbe
Copy link
Member Author

wtgodbe commented Aug 20, 2020

@BrennanConroy @halter73 just pushed 6e53284, any other concerns?

Copy link
Member

@BrennanConroy BrennanConroy left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

File issues for followup work?

@wtgodbe
Copy link
Member Author

wtgodbe commented Aug 20, 2020

#25086
#25087
#25088
#25089

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
area-signalr Includes: SignalR clients and servers
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add MessagePack support for the Java SignalR Client
5 participants