Native component to draw lines, bounding boxes, polygons, and keypoints

[avn3r-dn]

Problem

The self-driving car it's a nice tutorial that compares a model vs labeling data. This is a powerful tool to visualize how your model is doing and where it can improve. However, a more powerful statement of how Streamlit could be used for all Data related projects will be if I can create the labels directly on the app. In this case, the input is the model bounding boxes, then we can easily go over every box and fix any issues. This will allow Streamlit to serve both a platform for creating the Datasets and later visualizing how these datasets are doing. Making closer to end-to-end front-end solutions. I can already use Streamlit for prototyping classification labeling, adding drawing on top of images will allow it to do all type of CV problems.

Solution

1. Allow drawing on images
2. Draw Line, Bounding Boxes, Polygons and Keypoints
3. Return coordinates based on image location clicks based on image dimensions

Let me know if this something of interest for Streamlit.

Regards.

[avn3r-dn]

? my feature request had nothing to do with Bad Format why close?

[tvst]

Ooops, should have been #879 . Sorry about that!

[tvst]

As for this feature request, I think the best solution would be:

1. Write a separate library that draws those kinds of markers directly on numpy arrays and uses Streamlit to output the result.
   or
2. Wait for plug-in support to land (Plug-in architecture for Streamlit #327), and then implement this as a plug-in. The advantage here is you'd be able to also use JavaScript so draw markers and handle interactions (hover, click, etc).

And we'd be happy to help design either solution, if you are interested in developing it. We'd also link the library directly in our docs and wherever else it makes sense.

What do you think?

[andfanilo]

For info, aside for polygon this is mostly filled by Drawable Canvas

[ternaus]

It would be so convenient if there was an option to draw polygons...

I would like to write my own customized data labeling tool and streamlit looks like a good choice, except the situation with polygons.

[randyzwitch]

Closing, as Streamlit components and Drawable Canvas exists. Happy to re-open if I'm misunderstanding the use case.

https://github.com/andfanilo/streamlit-drawable-canvas

[jrieke]

Going to re-open because the drawable-canvas component is unmaintained now and we'd also love to have this as a native component. Please upvote with 👍 if you feel like this is useful!