Reduce intrinsic protocol level redundancy

[relu91]

As I have been requested to do during this F2F I'm creating issues related to my presentation about Thing Description redundancies.

In this issue, I want to focus on intrinsic protocol level redundancy (slide 7). In short, certain IoT protocols require common configuration parameters that are usually shared across all Web Thing form instances. This creates longer TDs and opens the door to copy&paste errors. This is not the first time that this issue pop up in fact we already have a good list of issues that somehow touch this same aspect:

• Describing initial connection #878
• Linking to top level initial connections #1242 Similar issue to 803.
• Introducing assumptions about TD instances that have no IP addresses #977 Sort of related. Deducing the IP address from an out-of-band way
• Single 'base' field but multiple 'forms' elements #803 Having the same feature as the security term in form protocols
• Allow more operations in a top level form #1070
• Add overridable global default value(s) for mediaType #204 (this one is relevant for Modbus protocol binding where we have the necessity to describe a common contentType for every form)
• Add overridable global default value(s) for mediaType #204 Media Type and Security Override: Sharing a media type and security (already possible) among forms
• Describing initial connection #878 Reused Connection: Implying a first connection that is not dropped but actually reused later
• Allow more operations in a top level form #1070 Reused Connection in WS: Similar issue to 878 but giving more details on WebSocket
• Describing a single WebSocket endpoint shared by multiple Things #1664 Reused Connection in WS 2: Similar issue to 878 abd 1070
• Reconsider the interaction between Thing::base, Form and how the protocol is signaled #1834Linking to Initial Connection 2: Similar issue to 803 and 1242

In the end, what is really missing is a global space to factor out protocol configuration and refer to those configurations in affordances only if needed. Sadly, this of course we'll make TDs more complex to read/understand.

[egekorkan]

I will use this issue as the umbrella for the initial connection discussion. @relu91 could you update the list with every issue listed at w3c/wot#1200. That way, people who stumble upon this issue have an easier life. If you are fine with it of course.

Everyone else, please refrain from commenting on this issue with ideas at this stage as multiple issues are already quite long.

[egekorkan]

(This comment will be edited. If you are joining the calls, please mention your opinion there to not clutter this issue. Each major point will be opened as an issue)

A summary of the discussions in the linked issues:

• There are 3 main discussions. Subpoint 2 is about a keyword (more or less), whereas subpoint 3 is more about the underlying mechanism. Probably subpoint 3 will be the major point of discussion since it includes lifecycle aspects.

  1. Having a place to put common connection information
  2. Linking to a common connection information.
  3. Signaling that a form in an affordance can reuse the connection established before so that the Consumer does not open a new connection per operation execution

• We want a global media type that behaves the same way as security in the root level, i.e. each form that does not have a contentType key, uses the one defined globally, somewhere in the root level of the TD. Relevant issues: Add overridable global default value(s) for mediaType #204 , BACnet media type wot-binding-templates#357 . Points to pay attention:

  • In case of affordances that are incompatible with the global media type, overriding must be mandated. An unstructured data at root level, e.g. jpeg, must be overridden by an affordance that has data schema.
  • Negative: Message serialization from TD consumption will be more complex as a preprocessing of the TD is now needed. If there are multiple global media types, referring to them and resolving those will increase the complexity of the preprocessing step.
  • Positive: It is easier check for the Consumer, which media types (their decoder/encoder) are needed to be supported to talk to the Thing. Furthermore, as can be understood in the BACnet issue, it is currently not clear that a form requires a media type. This addition can help with this.
  • These negative and positive aspects will be visible in other points and can be named "tradeoffs". As mentioned in Single 'base' field but multiple 'forms' elements #803 (comment), any optimization of TD instances create more processing need while making them easier to understand and write by humans, at least in simpler cases.
  • Linking aspect is common to single connection and single base points

• We want to support multiple base URIs that behaves the same way as securityDefinitions in the root level, where each form can contain a relative URI and refer to a base, like we do now with security. Relevant issues: Single 'base' field but multiple 'forms' elements #803, Describing initial connection #878 (starting at Describing initial connection #878 (comment))

  • Proposals are available in Describing initial connection #878: on the container structure called connections, endpoints: Describing initial connection #878 (comment)
  • Proposal about the container at Allow more operations in a top level form #1070 (comment)
  • Mention that canonical TDs can contain the expanded forms with everything doubled. This is applicable to global media type mentioned above.

• An initial connection to the Thing should be expressed somehow. This avoids misinterpretation of reopening a connection twice, when there is no need and gives correct semantics for some protocols. Relevant Issues: Describing initial connection #878, Introducing assumptions about TD instances that have no IP addresses #977, Allow more operations in a top level form #1070, Linking to top level initial connections #1242, Describing a single WebSocket endpoint shared by multiple Things #1664

  • Proposals are available in Describing initial connection #878 to:
    • have an op value called open or a signifier to establish a connection. Consensus from Sept15, 2021 is to not have an op. Describing initial connection #878 (comment)
    • dependsOn keyword in non-base forms
  • A proposal in Introducing assumptions about TD instances that have no IP addresses #977 shows URL templating to refer to the base connection: Introducing assumptions about TD instances that have no IP addresses #977 (comment)
  • Proposal in Allow more operations in a top level form #1070 (comment) for linking via pointers and has two proposals
  • The Consumer should be smart to not open new connections, even without this. This only makes it clearer for humans and simpler implementations.
  • There needs to be a way to signal a connection shared for multiple Things. See Describing a single WebSocket endpoint shared by multiple Things #1664.

• Wrapper Schemas comes into question if all communication over that initial connection follows the same message structure. Related issues Describing initial connection #878 (comment). This is relevant to global media type and security in a way. Also relates to data mapping.

[egekorkan]

I am done with the previous comment and will present the findings in the TD call today. For all points of discussion, there are multiple proposals that I put into this hackmd for now: https://hackmd.io/@egekorkan/r1lXdjwtA

[egekorkan]

In today's meeting, opinions on the past proposals were collected:

• Daniel: The simple case with one base and one protocol (and just HTTP), should not require complex forms or uri arithmetics. (+1 from Cris)
• Cris: These proposals are from a time where we wanted to be backwards compatible. Now we do not have the limitations and we should not think in that way.
• Cris: Proposal 4 is not good due to ordering problem.
• Cris: Proposal 5 mixed with 1 seems like a good direction.
• Luca: Security are connections, we can abuse that for TD 1.1
• Luca: Security and Connection can be put together but we should check compositionality
• Luca: Binding the base with connection is good but we can have a base without the connection.