Implement a single Websocket endpoint for things

[mishoboss]

Currently a UI or any 3rd party app needs to open as many WS connections as things there are in the gateway. Each WS points to a dedicated thing URI. E.g. like this:
const socket = new WebSocket('wss://mywebthingserver/things/robot', 'webthing');

However this brings difficulties and inefficiency. Also browsers by default don't allow a lot of simultaneously opened connections, though this is usually a configurable.

What I suggest is to introduce a single WS endpoint, with a slightly different protocol:
const socket = new WebSocket('wss://mywebthingserver/things', 'webthings');

Note that the protocol name is "webthings", not "webthing" and the endpoint is "/things".
The idea is this "webthings" protocol to absolutely mimic the "webthing" protocol, just add href everywhere like that:

{
  "messageType": "setProperty",
  "href": "/things/pi",
  "data": {
    "leftMotor": 100
  }
}

and

{
  "messageType": "event",
  "href": "/things/pi",
  "data": {
    "motion": {
      "timestamp": "2017-01-24T13:02:45+00:00"
    }
  }
}

[benfrancis]

I agree, but we don't need to define a different protocol with a different name. Just add this capability to the Web Thing WebSocket API.

See WebThingsIO/api#46

[mishoboss]

Maybe. Just, this URI is not needed for a native webthing device and won't make sense there. And having a dedicated "webthings" protocol will allow us to introduce some more gateway related stuff, like e.g. fetch all things as its done in the REST API., and aslo events on thing added, thing removed, etc.

[benfrancis]

Both the REST API and WebSocket API can be used by individual things, or gateways which expose multiple things. It's just that individual things don't implement the things resource.

[mishoboss]

Yes, sure. It's not a technical issue, but a protocol design issue.
And it will definitively make sense to extend the Things resource with some management actions and events, like I suggested above (thing added event, thing removed event). But I guess this is not on focus right now.

[hobinjk]

I think what makes the most sense to me is to have the Gateway serve a Thing description with a "things" member that contains the URIs (or TDs) of Things from /things. The WebSocket scoping then makes sense and can be more of a first class citizen of the API.

Events like "thing added" and "thing removed" would make sense as part of the thing type "gateway", i.e.

{
  type: "gateway",
  events: {
    thingAdded: {
      thingHref: blah
    }
  }
}

[benfrancis]

I really like the idea of giving gateways a Thing Description with their own web thing type and then have them provide a list of things they manage. In a way we're already treating the gateway as a thing by giving it actions, giving it a Thing Description and adding events as you describe would be a natural extension of that. I've filed WebThingsIO/api#91 to discuss that.

This would help resolve the debate about the webthing libraries serving one or multiple web things, because if any thing can be a gateway which links to other things then the webthing libraries can provide this as an option.

As suggested, this will also allow the gateway's Thing Description to include a link to a single WebSocket endpoint which can be used to monitor and control all of the things that it manages, including a Thing URL in each message to differentiate between things.

[YonasJ]

I would like to propose and extension to what mishoboss is proposing. Using some code snippets from the ESP32 Arduino driver I have been working on.

It seems to me to be redundant to mix and match the API's, especially since we are using service discovery. Rather than having the HTTP API advertise the WebSocket API, just advertise it separately.

Then the implementor could choose the API for their needs. The device I am working on has a button that you have to push, and the HTTP API is so laggy it cannot be used. Moreover, for a short push of the button the HTTP API would likely not ever find out about it. The WebSocket API works great, though.

But it is a big pain having one WebSocket API/URL for each thing.

If this were to happen, in my code, I would register both API's, like the following. And the WebSocket's API would be extended to be able to list the things/properties.

    // Register HTTP API
    MDNS.addService("webthing", "tcp", 80);
    MDNS.addServiceTxt("webthing", "tcp", "path", "/");

    // Register WebSocket API
    MDNS.addService("webthing", "ws", 80);
    MDNS.addServiceTxt("webthing", "ws", "path", "/");

Then we could add the two missing methods the the websocket's API so that a client could choose to use either web-sockets, or http.

My sense is that the greatest contribution of the mozilla iot project is a project that is easy to get started with, and a great protocol for describing the things (and so they can describe themselves).

If we handled things in this way, we could then also make it possible to support the COAP protocol. While the COAP protocol is quite similar to a hybrid of websockets and http, it has one major difference and that is the ability to keep trying to make sure a message is delivered. So, it could be essentially compatible with websockets/http, but add the reliable delivery option missing from websockets/http (what happens when there is a momentary network issue/the connection is lost).

    // Register CoAP API
    MDNS.addService("webthing", "coap", 1234);
    MDNS.addServiceTxt("webthing", "coap", "path", "/");

MQTT is another protocol that is popular, easy to develop for, but doesn't have much schema stuff. And may be worth supporting.

Then it would be possible to, using the original API suggestions above to make it so that all 4 transports can exist for the same JSON api.

For HTTP/COAP:

PUT /things/pi/leftMotors
BODY:
{
  "data": {
    "leftMotor": 100
  }
}

For WebSockets:

{
  "messageType": "setProperty",
  "href": "/things/pi",
  "data": {
    "leftMotor": 100
  }
}

For MQTT:

publish ("/things/pi", '{
  "messageType": "setProperty",
  "data": {
    "leftMotor": 100
  }
}')

And then each device could advertise the appropriate one(s). Since the API would be the same for each, it would be easy to support multiple protocols for devices with lots of RAM, and for low RAM devices or cases where message delivery needs to be ensured CoAP or MQTT would be a boost.

It would also be a boost for CoAP and MQTT to benefit from the schema stuff available in mozilla-iot.

Since the messages and data are equivalent, implementors could choose the underlying protocol, but get all the benefits of the schema for iot things.

[benfrancis]

Thank you for the interesting suggestions, @YonasJ

I agree that it would be nice to have a CoAP binding of the Web Thing API. I imagine it to be much like the HTTP binding, but making use of the OBSERVE option on GET requests to push property changes, action status changes and events to the client. Like you say, it could be a lightweight alternative which combines benefits of both the HTTP and WebSockets bindings. A web thing could serve a separate Thing Description over CoAP with coap:// URLs in links (perhaps with a CBOR rather than JSON encoding), or a single combined Thing Description could provide both http:// and coap:// URLs for each resource.

When it comes to the WebSocket binding I actually think it makes a lot of sense to advertise the WebSocket endpoint in a Thing Description served over HTTP. WebSockets are really an extension of HTTP and you have to send an initial HTTP GET request before upgrading that request to a WebSocket anyway.

As for MQTT, that is not really a web protocol and doesn't have an official URI scheme so doesn't lend itself well to the Web of Things. An adapter which bridges MQTT to WebSockets (or CoAP) might be a better approach.

I definitely agree we should support multiple web things per WebSocket.

[YonasJ]

Thanks for your comments.

Upon reflection, I agree with what you are saying about MQTT and mixing the HTTP and WebSockets.

What happens next, can I just modify the WebThingsAdapter to support a single websocket url that uses the JSON format proposed above and make a pull request?

Or is there somebody who would make a change like that?

Or is there other people we would need to discuss it with?

Thanks.

[benfrancis]

I think the next step is to specify (in WebThingsIO/api#46) how the Web Thing API should accommodate multiple things per WebSocket. Once that is in the spec we can implement the spec in our gateway implementation.

[mrstegeman]

Fixed by #2085