Add an element in the mediaFile block for Entity Lists in the OpenRosa manifest to specify a URL to get Entity information

[lognaturel]

As the implementer of a data collection client, if there are Entities in the client's representation that are not included in the server response
I want to be able to request the status of those specific Entities
So that I can delete any Entities that no longer exist on the server

As the implementer of a data collection client
I want to be able to get information about specific Entities
So that I can make sure their state is up to date in my local representation

See Collect issue getodk/collect#6231 describing desired XML API.

Strawman proposal

• Add an integrityUrl element that may only appear in a mediaFile block with type attribute (see Identify attached entity lists in the manifest response #665) set to entityList.
• The integrityUrl value is a URL that accepts a query parameter named id representing the UUID of an Entity that the client wants to know about. A comma-separated list of UUIDs is also accepted.
• The API endpoint returns an XML response as described here / Locally created entities removed from the server should eventually be removed locally collect#6231

Interesting cases to design for

Deleted and purged Entities

Currently purged Submissions are completely removed so it's possible to reuse an id. Entity purge is not yet implemented. We'll need to decide whether to keep historic UUIDs or allow reuse. Allowing reuse would only be an issue in very rare cases.

Let's not allow reuse of UUIDs when Entities are soft-deleted to avoid the case where Entities with the same UUID are in trash and active.

Rejected submissions that never resulted in Entity creation

Two concepts have emerged so far: send the instanceID to the integrity API to detect this case or keep a notion of "phantom" entities that wanted to be created but never were.

Todo

• Implement strawman
• Use in Collect
• Consider max URL size and whether query parameters are appropriate
• Consider naming of deleted element so the name matches what it represents (true: server knows of the Entity and it shouldn't be on that client; false: server either doesn't know of the Entity or it knows it and it should stay on the client)
• Post on forum and iterate on element names + response structure

CC @seadowg

[seadowg]

The integrityUrl value is a URL that accepts a query parameter named uuid representing the UUID of an Entity that the client wants to know about. A comma-separated list of UUIDs is also accepted.

I always trend towards feeling that UUIDs should just be referred to as ids rather than explicitly passing them around as uuid. Is there a good argument for that not being the case?

[lognaturel]

Is there a good argument for that not being the case?

For better or worse, we already have several APIs that use the name uuid for the unique ID of an Entity. This one isn't user-facing so I don't feel strongly about it but I do generally think consistency is helpful. If we could go back in time I would probably have preferred we use __id consistently everywhere.

[matthew-white]

I always trend towards feeling that UUIDs should just be referred to as ids rather than explicitly passing them around as uuid.

We ended up deciding to name the query parameter id rather than uuid. I've updated the issue description. Central uses the name uuid in its REST API, but in XForms, the name id is used. We ended up concluding that we should use the name id wherever there is XForms/XML/OpenRosa. Related: #679

[matthew-white]

• For v2024.2, the main case we need to address is soft-deleted entities, which are still present in the database and easy to look up in the entities table.
• Once we support purging entities, we'll need a way to track the UUIDs of entities that have been purged.
• We may also implement something related to submission approval, but probably not until a later release (see Tell clients to delete entities when their submissions have not been approved #685).