Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Treat tensor values as instances #1756

Open
emilk opened this issue Apr 3, 2023 · 0 comments
Open

Treat tensor values as instances #1756

emilk opened this issue Apr 3, 2023 · 0 comments

Comments

@emilk
Copy link
Member

emilk commented Apr 3, 2023

We want to be able to address individual pixels in an image, and values in a tensor. For instance: click on a image pixels to select it.

One way to do so is to use our InstanceIds as a pixel index.

This means log_image would be similar to log_points, and we would never have something like log_images.

We would then be able to have multiple values per pixel. For instance: the same tensor could both have a Density component and a Confidence component. This could replace our special-casing of segmentation images.

We would need to store the tensor shape as some sort of meta-data of the Entity, perhaps as a "mono-component" (new concept) of a multi-instance entity.

We would also get sparse tensors for free. A sparse bit-map (e.g. a connectivity graph) could be stored using a zero-sized component.

Before we can work on this we might need to fix the component-converters (https://github.com/rerun-io/rerun/blob/main/design/component_datatypes.md) first, in order to handle JPEG-encoded tensors.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant