APP-8003 Add world state store service

[DTCurrie]

Introducing a new service type, the WorldStateStoreService.

This service will be used by visualization modules to return information about the machine's world object store to be rendered client-side.

List of changes:

1. Update the common Pose message to include a label
2. Add uuid and metadata to the common PoseInFrame message
3. Add a new common message GeometryInFrame, which is similar to the PoseInFrame message so we can use both consistently
4. Add Points and Line geometry types and added them to the oneof geometry_type on the Geometry message
5. Introduce the WorldStateStoreService with three rpcs:

• ListWorldStateUUIDs returns all world state object uuids
• GetWorldStateObject returns a world state object by uuid
  • oneof world_state_object will be either a PoseInFrame or GeometryInFrame
• DoCommand sends/receives arbitrary commands

Scope doc: https://docs.google.com/document/d/1ionvyBa7x3HZU_rwDPBvrAUrQjBrP6sJ10Z_xbBTue8/edit?tab=t.0#heading=h.tcicyojyqi6c

[DTCurrie]

Removing some folks from review while I tweak things to avoid noise.

[raybjork]

Was there a scope for this? Would be helpful to understand the thinking behind it

[DTCurrie]

Was there a scope for this? Would be helpful to understand the thinking behind it

Yes sorry, meant to add a link in the description. Here you go: https://docs.google.com/document/d/1ionvyBa7x3HZU_rwDPBvrAUrQjBrP6sJ10Z_xbBTue8/edit?tab=t.0#heading=h.tcicyojyqi6c

The API design has been slightly tweaked based on testing @micheal-parks did, but for the most part things are the same.

[raybjork]

This looks good to me from a code perspective based on what was decided on in the scope doc. I wonder though if it would be possible to put this under the app proto instead of the general service proto. I expect this API to change frequently and it seems like there is a lot more leniency towards making breaking changes to the App API as compared to resources

[DTCurrie]

This looks good to me from a code perspective based on what was decided on in the scope doc. I wonder though if it would be possible to put this under the app proto instead of the general service proto. I expect this API to change frequently and it seems like there is a lot more leniency towards making breaking changes to the App API as compared to resources

Fair point, I brought this up to steve and he suggested it could live here because it is a new service resource type, but we can mark it as experimental for now, and allow some level of leniency until we have published some Viam-supported visualization modules and are confident with the API.

@micheal-parks has also been testing visualizations with the assumption that this is how the API will work, so we have some real-world proof of concepts to work from. I ended up writing the code to implement the fake model for testing in the RDK. I can work with micheal to test these APIs using his current code and ensure they work as expected, then report back on the results.

I'd be happy to get some more opinions on this before moving forward, though, if you have anyone in mind.

[DTCurrie]

Note from @erh to use existing world object state

[micheal-parks]

There are a few minor problems - and one bigger problem - with using the existing GeometriesInFrame struct:

Minor issues:

• The geometry does not have a bytes uuid (can we add?)
• The geometry struct does not have a metadata object for drawing poses, colors, etc (can we add?)
• We have sphere, box, capsule, but would need points and lines to encompass what people draw
• Some things that people draw are not geometries (like poses represented as arrows)

The bigger issue is:

• ListWorldObjects was designed to return an array of uuids packed into bytes so that rapid inexpensive polling for store state could be done from a remote client
• The client decides which uuids are new and fetches geometry details only for added items instead of fetching complete details for all world objects on each poll
• An array of world states would be an array of arrays, and with this data structure it's easier to pack items into the GeometriesInFrame instead of one object per world state which doesn't make much sense
• This now makes it impossible to diff on the client
• People often draw hundreds or thousands of objects with this tool - it's not tenable to fetch that much data per poll

We could just make this a geometry list store if the minor issues are solved, but IMO it's still weird because geometries are only some of the things people visualize

[DTCurrie]

There are a few minor problems - and one bigger problem - with using the existing GeometriesInFrame struct:

Minor issues:

• The geometry does not have a bytes uuid (can we add?)
• The geometry struct does not have a metadata object for drawing poses, colors, etc (can we add?)
• We have sphere, box, capsule, but would need points and lines to encompass what people draw
• Some things that people draw are not geometries (like poses represented as arrows)

The bigger issue is:

• ListWorldObjects was designed to return an array of uuids packed into bytes so that rapid inexpensive polling for store state could be done from a remote client
• The client decides which uuids are new and fetches geometry details only for added items instead of fetching complete details for all world objects on each poll
• An array of world states would be an array of arrays, and with this data structure it's easier to pack items into the GeometriesInFrame instead of one object per world state which doesn't make much sense
• This now makes it impossible to diff on the client
• People often draw hundreds or thousands of objects with this tool - it's not tenable to fetch that much data per poll

We could just make this a geometry list store if the minor issues are solved, but IMO it's still weird because geometries are only some of the things people visualize

I noticed some of these issues and was thinking about how best to address them. We could flatten the world states into a single world state, then you have a single list of geometries and transforms to deal with. We would then have to add the fields you mentioned to geometries. I am a little hesitant to make alterations to that common proto to support this one service type, though. And that still has to introduce a new WorldStates type or something.

I may revert the changes, then we can sync over email to decide on this quickly.

[DTCurrie]

Updated per latest conversation:

1. New message type: GeometryInFrame, which is just one geometry instead of a repeated list
2. Add uuid and metadata to PoseInFrame and GeometryInFrame
3. Add label to PoseInFrame
4. Add Line and Points to Geometry['geometryType']. They have one field: array = Float32Array. This is essential for faster over-the-wire speeds, especially for point clouds
5. The world state store service is a list of GeometryInFrame / PoseInFrame

[raybjork]

Ok after reading deeper I understand more. I think initially I was just surprised by everything that was changing because that was not my read from the email for how the changes would need to be written.

I think all we need to do here is add the metadata and UUID onto the GeometryInFrame object since PoseInFrame is a GeometryInFrame with a nil geometry_type (we could alternately do it on the geometry itself, I'm not sure if this would be better). Then the GetWorldStateObjectResponse can just become only the GeometryInFrame