More Communication Protocols

[djungelorm]

The underlying server communication protocol should be moddable, to allow communication via other means than just google's protocol buffers.

Then we can support, for example:

• http://en.wikipedia.org/wiki/SOAP to make it easy to interact with kRPC from web apps (e.g. JavaScript)
• CCSDS protocol???

[thesamprice]

Send me an email, I could probably help with the javascript / CCSDS protocol stuff.
My C# skills are pretty bad, but I know javascript,c, python.

[djungelorm]

This is fairly low on my priority list right now, but when I do get round to it I'll put together a prototype and if you could try out the SOAP interface from JS that'd be great (I don't know JS very well...)

I was thinking we could allow additional DLLs to be placed in kRPC's plugin directory, which would provide additional communication layers. The in-game server interface could then allow you to choose which communication protocol you want the server to use.

[kleiram]

Quick suggestion: REST instead of SOAP, since it's also HTTP but a lot easier to interface with!

[Outurnate]

GRPC might be cheap to generate code for

[sgayda2]

GRPC would solve a lot of your issues, you are already doing what grpc does, proto messages over ipc but it will do a lot more as well, and make it more reliable and stable.

[gitanat]

Seconding GRPC, it seems well maintained and solves the problem of having to write client libraries.

Unfortunately it won't help for getting data in the browser, as GRPC has no defined mechanism for interacting with it (like websockets).

[djungelorm]

I've only had a quick look at GRPC, but here are two issues that came to mind compared to the current server protocol:

• I'm not sure GRPC would allow for kRPC "streams". GRPC has "streams" but they sound a bit different as "the client completes once it has all the server's responses."
• I don't think dynamic clients are possible with GRPC? The kRPC Lua and Python clients currently get a list of available RPCs from the server and create the stubs dynamically. This keeps their implementation nicely separated from the server.

Also, I've started working on making the server protocol used by kRPC extensible. My vision is that the user will be able to start one or more servers, and each server can be of a different 'type'. I see no reason why GRPC couldn't be included as one of these 'server types'. Then GRPC can even run alongside the current server protocol (with its streams and dynamic client support). A REST 'server type' is also planned.

[djungelorm]

In case you're interested, the changes are on this branch: https://github.com/krpc/krpc/tree/feature/server-refactoring

[sgayda2]

I have used GRPC streaming in a project and it worked by having the client connect to the server to get data. The server would never finish sending data so the stream was maintained until the client decided to end the connection.

[gitanat]

I don't think dynamic clients are possible with GRPC? The kRPC Lua and Python clients currently get a list of available RPCs from the server and create the stubs dynamically. This keeps their implementation nicely separated from the server.

Indeed, GRPC does force you to have a well defined interface, and compiling protobufs at runtime is not easy.

Is that added flexibility useful in practice, though? Doesn't the current code have to agree on the protobufs ahead of time anyway?

If we do get gRPC, REST would come for free, if we are willing to run a external program

[djungelorm]

Yes the current code does have to agree on the protobufs, but not the RPCs. The protobufs just define what a request/response message looks like, not what the RPCs are. So you can add/change RPCs on the server and the python/lua clients automatically pick up the changes.

The existing C++, C# and Java clients have generated stubs though, so these could be just be replaced with gRPC.

[Lokaltog]

Let me know if you need help testing a websockets/REST API, I'll definitely port KeRD to kRPC as soon as this is ready.