Add support for defining and using regions-of-interest (ROIs)

[niksirbi]

Overarching theme

One of the key elements of the project's long-term vision is:

Integrate spatial data about the animal’s environment for combined analysis with motion tracks. This covers regions of interest (ROIs) such as the arena in which the animal is moving and the location of objects within it.

Having a representation for ROIs is crucial for many useful and popular metrics such as:

• time spent in specific areas
• number of entries/exits into specific areas
• relative orientation of an animal with respect to an object in its environment

Specific aims

To support this goal, users should be able to define ROIs either by passing coordinate values, or ideally, by drawing them on top of the video/frame.

On the Python API side, users should be able to define objects such as points, lines, polygons (and maybe also circles, ellipses, and rectangles, for convenience) by passing vertex coordinates or the geometric parameters of each shape (e.g. for the circle this could be centre + radius).

Ideally, we should rely on existing mature solutions as much as possible, for example, the Shapely Geometry classes, with thin convenience wrappers on top of them (if and as needed).

Additional context

Related to #368 and #370.

This issue is meant to track the data structures and API interface for ROIs.
The drawing of the ROIs via a graphical user interface is a separate issue #378.

[willGraham01]

We should brainstorm the core features that RoIs need to have - this will enable us to design the class skeleton from this. Features that are critical for RoIs to have that I can think of off the top of my head (and further discussion is welcome):

• Need to be easily user-definable. My gut says that the user should be able to give a series of points (in some sensible format) with the option to either "close" the shape (creating a polygonal region) or "open" (creating a line, EG a wall).
• The above creation method should consider how a user working in the napari plugin will create an RoI (Provide a widget for drawing regions-of-interest (ROIs) in napari #378) though this is currently not in the scope of this issue.
• An RoI should be able to determine if a position is inside the RoI (static in time)
• An RoI should be able to determine if a keypoint entered / exited the RoI between two frames.
• An RoI should be able to determine the displacement to another point (or from a point to it). This will provide the functionality for Compute distance between a point and a line #368 and Compute angle between a vector and a plane perpendicular to the image plane #370. (shapely does this off the bat, hence the appeal)
• The above features, where appropriate, should be broadcast-able across a time axis (and potentially an individuals / keypoints axis too)

We should also investigate existing (mature) libraries that support representations of geometric data. I had a quick play with shapely as suggested above - and was quite fond of it - but would need to do a detailed check as to whether it would be suitable for our needs above. Alternatives are welcome, since I don't think we want to be supporting shape-representation ourselves (from the ground up).

As a suitable roadmap:

• Create (and report here) a design for the RoI class.
• Implement an RoI class that can be instantiated (Introduce basic classes for regions of interest #396),
• ROI can determine if a point lies within it (if a polygon). RoI should be able to determine if points lie within them #408
• Implement features necessary for Compute distance between a point and a line #368 and Compute angle between a vector and a plane perpendicular to the image plane #370. Note these should be broadcast-able across other dimensions as appropriate (time, individuals, keypoints). Add functionality for computing ego- and allocentric angles to RoIs #416
• Implement time-dependent features, like "An RoI should be able to determine if a keypoint entered / exited the RoI between two frames". This will need to be considered in conjunction with Support event arrays #369, but if we stay true to ensuring RoIs broadcast across dimensions, we should be in good shape for creating such arrays. Condition arrays: compute region occupancy #421
• Create an example for the website (and for Sepi to potentially test?). Example for Boundary Angle Calculations #440
• Consider the napari widget implementation Provide a widget for drawing regions-of-interest (ROIs) in napari #378. Though this will likely take a backseat to Support event arrays #369 in the event there is time pressure.

[willGraham01]

Had a play with shapely in #390, and with a bit of effort managed to get some basic wrappers written.

Main difficulty is that shapely classes are a pain to subclass, so ended up having to write wrappers. But since shapely boundaries return other shapely classes, had to also provide functionality for overwriting the return value of those methods with our own. But now that's done, it should be quite simple to implement the above features within these wrapper classes (in particular, implementing them in a way that is broadcastable across dimensions of xarrays).

[niksirbi]

Thanks for outlining this plan Will, I fully agree with it.
I'll take a look at #390 in the following days and share my thought on the draft implementation.