Reorganize and consolidate work items

[egekorkan]

fixes #24

The rendering below seems to have stopped. Meanwhile use this:

https://htmlpreview.github.io/?https://github.com/w3c/wot-charter-drafts/blob/f521e06bb97ba5eee19a58387ebda17b1977294c/wot-wg-2023-draft.html

---

Preview | Diff

[benfrancis]

Feedback: #24 (comment)

[mlagally]

In general I think this removes too much text and puts the entire focus only on TD 2.0.
I would expect to call out whether TD 2.0 will have breaking changes, and call out significant new requirements that are going to be addressed by this specification.

The current list of suggested enhancements rather sound like TD 1.2.
To revisit the discussion in yesterday's use cases call, I suggest we collect these proposals in a separate document, do not delete contributions by other members, but rather discuss prioritization of the tasks together in the WG.

[mlagally]

This is a rather vague description - can we please get it more precise?

[egekorkan]

I wanted to add some intro sentence to each work item. The meaning of this sentence is the following one.

[mlagally]

Which use cases are you referring to?
Not sure what is meant with "repeating payloads"?

[egekorkan]

This is basically when there are repeated payloads and this results in repeated data schemas in TDs. You can see this in ecosystems/standards like ECHONET Lite Web API, Philips Hue or the HTTP Baseline Profile which requires repeated payloads for actions.

In the case of HTTP Baseline Profile TDs, the additionalResponses is used which is sort semantically wrong. Additionally, even if we have schemaDefinitions in the Thing-level, we do not have a way to point to the "relevant" part of the payload for that affordance.

[benfrancis]

I had assumed that "repeating payloads and connections" was in reference to protocols like WebSockets which currently can't be effectively described in a Thing Description. For that I actually prefer "Connection and Payload Reuse" from the current text, although in that case the connection re-use is the important thing. I think the payloads for WebSockets have to be defined in a subprotocol.

@egekorkan wrote:

In the case of HTTP Baseline Profile TDs, the additionalResponses is used which is sort semantically wrong. Additionally, even if we have schemaDefinitions in the Thing-level, we do not have a way to point to the "relevant" part of the payload for that affordance.

The issue with action queues is not really "repeated" payloads and we already have operation names for describing "manageable actions". The missing pieces are:

1. Being able to describe dynamically created resources
2. Being able to define a data schema for a response to an invokeaction request which is separate from the action output schema

The latter is an example of a much broader problem which is that the TD spec needs to be much clearer about how to define data schemas for each type of operation, including meta operations (see w3c/wot-thing-description#1665 for a detailed breakdown).

If this all needs to be summarised in a single sentence I would personally word it something like:

"This includes describing dynamic resources, re-usable connections, historical values and more precise definitions of data schemas for each type of operation."

---

Side note: Thing Descriptions using the HTTP Basic Profile do not use additionalResponses or schemaDefinitions to define action payloads, the ActionStatus payload format is defined in the protocol binding specification in the profile and is assumed by conformant Consumers.

[egekorkan]

@benfrancis I will change it based on your comments. I agree with them.

Side note on the side note: https://github.com/w3c/wot-testing/blob/main/data/input_2022/Profile/TD/Oracle/TDs/BluePump%20WebThing.td.jsonld is using those terms

[benfrancis]

Side note on the side note: https://github.com/w3c/wot-testing/blob/main/data/input_2022/Profile/TD/Oracle/TDs/BluePump%20WebThing.td.jsonld is using those terms

🤦 That extra metadata is completely redundant. I guess there's no reason you can't provide it, but it's only useful for Consumers which don't implement the profile. I would expect Consumers conforming to the profile to ignore it.

The only reason we were trying to describe the protocol binding declaratively in a Thing Description was to illustrate it as an informative example Thing Description in the WoT Profile specification showing how you'd describe to the same protocol binding without the profile (similar to how the WoT Discovery specification using a Thing Model to describe the Directory Service API), which turned out not to be possible.

[benfrancis]

Note: It may also be necessary to add new operation types to the next version of WoT Thing Description, like subscribemultipleproperties, observemultipleproperties, observeaction, observeallactions etc., but that might be covered under the general topic of "increasing the descriptive power of TDs". I'm assuming the list is not exhaustive.

[danielpeintner]

The current list of suggested enhancements rather sound like TD 1.2.

I don't think anyone is looking for TD1.2 version since this puts as again in the pandora box of backwards compatibility issues. We definitely aim for 2.0!

[egekorkan]

@mlagally I agree with @danielpeintner above. We have so many issues and even PRs in the TD that break backwards compatibility, we cannot just do a TD 1.2.

As discussed in the first charter discussion sessions, the specs get issues that are not simply a use case. We cannot work on a spec purely by the use cases, the use cases can and should bring additional requirements though.

Regarding deleting other contributions, my task was to do exactly that. If you do not want your work item to be consolidated, please say it specifically.

[mmccool]

My general feeling is that this PR is not really a bottom-up consolidation of existing detailed work items but a proposal for a set of top-down scope categories. That's ok, but we should discuss them as such and merge them with the set of top-down "summary" items I introduced yesterday. Some of the above discussion, e.g. handling websockets, etc. should really be in a detailed work item, and in the details document, not in the charter. I think the scope summary itself needs only a general statement like "Improve the ability of the TD to describe additional protocols."

[egekorkan]

@mmccool I am not sure if your comment is looking for something else in my PR or is it asking for future improvements in another PR?

I am not sure if you want to have any work items at all. If you want me to align it with the scope you have defined above, I think that they are more or less the same thing so they would be redundant.

[mmccool]

What I think we should do is take the proposed high-level items out of this PR and create another PR that merges them into the high-level summary already in the document, after merging #64 to get rid of the long list of detailed work items. The merge may still work after that PR but my guess is there will be conflicts and we will probably want to consolidate the formatting, etc. It also seems there are some detailed issues raised above we will have to address.

Let's do that merging live in the next meeting (I would call it a win if we can get the scope section out of the way in the next meeting) so we can tweak the wording and formatting (i.e. using a DL rather than a UL if that's what we want, but using a DL implies we need a "short name" for each category of work).

[mmccool]

Suggest merging and reorganizing as follows:

• Updates to existing specifications:

  • Address major issues discovered during implementation of already published specifications. Unlike the last charter, these would not be limited to backward-compatible updates.

• Support WoT integration:

  • The WG will improve the usability of WoT in other ecosystems and communities. Thus, the WG will define core binding and profiling mechanisms, specify additional protocol and payload bindings, and define appropriate profiles.

• Improve management of TDs:

  • The Thing Descriptions require additional mechanisms around them for their management. For example, the working group will update the Discovery specification with additional features, including geolocation. In other specifications, the WG intends to add a mechanism to sign TDs, define how to manage versions of TDs and define how to parse, validate, consume and lint TDs.

• Increase descriptiveness of TDs:

  • The WG will work on increasing the descriptive power of TDs to allow using them in more use cases. This includes supporting historical values (timeseries), repeating payloads and connections, support both static and dynamic geolocation information, and manageable actions.

• Improve security and privacy:

  • Define reusable, external security vocabularies, simplify the use of security schemes, and define and onboarding lifecycle process to establish mutual trust and identification, and other work as appropriate.

• New Use cases:

  • Address the functional requirements of new use cases, such as Digital Twins.

[benfrancis]

My suggested refinements to @mmccool's proposal...

Updates to existing specifications:

• Address major issues discovered during implementation of already published specifications. Unlike the last charter, these would not be limited to backward-compatible updates.

• "Address major issues" -> "Add new features and address issues"

(#52 (comment))

Support WoT integration:

• The WG will improve the usability of WoT in other ecosystems and communities. Thus, the WG will define core binding and profiling mechanisms, specify additional protocol and payload bindings, and define appropriate profiles.

• "Support WoT integration" -> "Support WoT interoperability"
• "The WG will improve the usability of WoT in other ecosystems and communities" -> "The WG will improve out-of-the-box interoperability and enable the integration of WoT into other ecosystems and communities"

(#24 (comment))

• "protocol and payload bindings" -> "protocol and payload binding templates"

(Pending conclusion of the discussion in #14)

Improve management of TDs:

• The Thing Descriptions require additional mechanisms around them for their management. For example, the working group will update the Discovery specification with additional features, including geolocation. In other specifications, the WG intends to add a mechanism to sign TDs, define how to manage versions of TDs and define how to parse, validate, consume and lint TDs.

• "The Thing Descriptions require additional mechanisms around them for their management." -> "" (remove this sentence since it's just repeating what it says in the heading).

(#55 (comment))

Increase descriptiveness of TDs:

• The WG will work on increasing the descriptive power of TDs to allow using them in more use cases. This includes supporting historical values (timeseries), repeating payloads and connections, support both static and dynamic geolocation information, and manageable actions.

• "This includes supporting historical values (timeseries), repeating payloads and connections, support both static and dynamic geolocation information, and manageable actions." -> "This includes describing dynamic resources, re-usable connections, historical values and more precise definitions of data schemas for each type of operation."

(#55 (comment))

Improve security and privacy:

• Define reusable, external security vocabularies, simplify the use of security schemes, and define and onboarding lifecycle process to establish mutual trust and identification, and other work as appropriate.

• I have some concerns about how compatibility with plain JSON consumers will be maintained if consuming mandatory security metadata requires reference to external security vocabularies (TD: Ensure security metadata can be parsed as plain JSON #57). It would be good to have a preliminary consensus on how that would be achieved before committing to this specific detail in the charter.

• It's still not clear to me what "onboarding" would involve in practice. A more concrete example in the details document might help.

New Use cases:

• Address the functional requirements of new use cases, such as Digital Twins.

I think this is a good way of putting this.

[egekorkan]

@mmccool I have initially copy pasted your feedback and incorporated some points of @benfrancis . Ben, I have not incorporated everything and I have checked boxes accordingly. I will have another pass at this later since I have to stop for today.

[egekorkan]

@benfrancis @mmccool I have went through them again and did some small changes:

• Aligning capitalization
• Removing names of deliverables after the title of the workitem as Michael mentioned in the last meeting. Correspondingly adapting the paragraph just before.
• Aligning use of acronyms

@benfrancis : I could not find specific solutions to the boxes I did not check. However, #14 and #57 are tracking those. Regarding, onboarding another issue is definitely needed. I have the same question as you.

[benfrancis]

• I have some concerns about how compatibility with plain JSON consumers will be maintained if consuming mandatory security metadata requires reference to external security vocabularies (

If people are OK with the idea that protocol binding templates which define security schemes must also define a JSON-LD-style prefix which TDs must always use to reference its ontology, then I think this is OK. It's not ideal because the Thing Description specification would no longer be self-contained for HTTP-based use cases which follow the defaults in section 8.3.1, but it could at least be workable. See #57 (comment)

[benfrancis]

@egekorkan wrote:

Regarding, onboarding another issue is definitely needed. I have the same question as you.

OK, I've filed #67 for that.

It would be good to get feedback from more people on #14 (which talks about how protocol binding (templates) and profiles might fit together) and #57 (ensuring that security metadata can still be consumed as plain JSON).

BTW, as I understand it @mmccool's idea was that the scope overview you are writing would replace rather than add to the existing bulleted summary at the top, though I see why you might want to keep some items separate.

[mmccool]

• I have some concerns about how compatibility with plain JSON consumers will be maintained if consuming mandatory security metadata requires reference to external security vocabularies (

If people are OK with the idea that protocol binding templates which define security schemes must also define a JSON-LD-style prefix which TDs must always use to reference its ontology, then I think this is OK. It's not ideal because the Thing Description specification would no longer be self-contained for HTTP-based use cases which follow the defaults in section 8.3.1, but it could at least be workable. See #57 (comment)

I think we have to define mandatory prefixes for many reasons, including validation and signing. I am not against it. It might violate some JSON-LD principles but I can live with that. We can define them in a registry, and then when a Consumer (or a validator, like a Directory) sees a predefined prefix it can assume the associated (and fixed) vocabulary without downloading anything. We should also state that if someone wants to use an extension, they should define and use a consistent prefix that does not conflict with an existing one. We could even insist on a naming convention for prefixes not mentioned in the specs, like "exp-myprefix", so "experimental" extensions would be obvious.

We can do this two ways: define security scheme vocabulary as part of a protocol binding, or in separate vocabularies. If security schemes are closely associated with protocols the former might make sense, but we can defer that decision. It might also be possible to define some "generic" security schemes (like "password" or "token") that get mapped by protocol bindings to particular protocol-specific schemes. Others like "apikey" that are carried in payloads or in URLs are already generic so we know that not ALL security schemes will be protocol specific. But we can defer the details until the work commences in the new charter...

[mmccool]

None of the previews seem to be working correctly and when I look at the files I don't see the changes I expected either. Well, I hope we can sort it out by the meeting...