Make Widget.useblit robust against underlying canvas changes

[anntzer]

As noted in #30490 (comment) (and #30485, the original issue, also #25075), Widget.canvas, which is typically initialized as self.useblit = useblit and canvas.supports_blit, is fragile against changes in the underlying canvas (the most common case in practice being a temporarily switch from a canvas that supports blitting to one that doesn't, when saving to vector output).
@tacaswell wrote some code to handle that in #25085, but some further fixes were necessary in #30490 which uses a different approach; a unified way of handling the issue would be nice to have.

Status:

• Button
• _SelectorWidget
• SpanSelector
• ToolLineHandles
• ToolHandles
• RectangleSelector
• LassoSelector
• PolygonSelector

Not supported yet:

• CheckButtons
• RadioButtons
• Cursor
• MultiCursor
• Lasso

[tacaswell]

It is possible that my approach in #25085 was too complicated and #30490 is the better path is all cases.

On the other hand, if we land #29855 we can end up getting into some surprising cases and need something more like #25085 everywhere.

[timhoffm]

Anticipating that #29855 will come, #30490 is too simple.

[timhoffm]

Does somebody know how blitting works exactly? From a quick look it seems that we sometimes store a background image, and sometimes don't and just draw a selected region. But maybe I'm mistaken.

Would it make sense to move the blitting infrastructure to the canvas (e.g. storage of background information)? That way, we directly remove the outdated information when switching the canvas.

[QuLogic]

Moving blitting to a more centralized location is certainly something we'd like to do, see #22183 for example.

Exposing it to a backend as some kind of overlay may also be a useful thing to do, though I'm not sure that's possible right now in the current architecture.

[timhoffm]

Ok, so the basic idea is

# standard draw of artists A ... E
draw(A, B, C, D, E)

# store background image
bg_image = draw(A, B, C)

# blitted draw:
restore(bg_image), draw(D, E)

Some questions/observations:

1. is it always the case that we restore the bg_image or do we have to consider cases, where we just draw on top of what is already
   there? For example a fully opaque image could be drawn on top of a previous one without the need to restore the background. But possibly background restoration is cheap enough so that we can always do it.
2. There can only be one background image. Therefore, the image should be owned by a single instance (either the figure or the canvas). It should not be stored on an Artist (like we do for widgets). It should also not be free floating like in the Minmal Example, because we may need to invalidate it.
   Edit: This doesn't seem to be true. Animation._blit_cache uses separate background images per Axes.
3. The animated flag is used to determine whether Artists are background or not. We can continue to use that, even though the naming is not great (something like blit_foreground would be better), but that's a detail and we can consider a renaming/alias later.
4. The authority of the blitted drawing logic should be with the figure, because it has access to the artists.
5. Do we need a flag Figure.use_blit?. If false a regular draw is performed, if true we use blitting. This distinction may be needed because blitting induces a separation in foreground and background and thus may affect the draw order. - Or can we live without that because by default all Artists are not animated and therefore all artists are in one group.
6. We may have two helper functions Figure.draw_blit_background, Figure.draw_blit_foreground.
7. Is this a good approach?

   draw():
       if background_exists():
           restore_background()
           draw_blit_foregound()
      else:
           draw_blit_background()  # implicitly or explicity store that
           draw_bilt_foreground()
   

   Then, deleting the stored background would result in a full draw the next time. - Can be used when switching the canvas - or also manually if you want to change something in the background.

[anntzer]

I just realized that this issue is also linked to #23882 (see also the linked discourse topic): it would be reasonable to make copy_from_bbox() return a standardized type (essentially, a pair of the raster image and its (x, y) position), and reimplement restore_region() in terms of draw_image() (modulo a transparency problem issue mentioned in the linked issue).
By doing so, we stop having to worry that different canvases have different background formats, and can furthermore require that restore_region() be defined on all backends (whereas copy_from_bbox() remains undefined on vector backends).

[tacaswell]

to point 2, because restore_region can restore only a region of the canvas, the constraint is a bit messier in that each pixel should only be in one background. Thus if you have two blitting cursor widgets each in their own Axes and they only cache the axes the work on everything mostly works most of the time. When you have two widgets on the same axes both blitting is when the chaos happens.

Doing blitting also currently requires a draw_event callback to re-draw the blitted artists when the figure is resized etc.

Puling all of this up into something that lives on the figure and then having the figure know which artists are blitted (rather than the artist knowing if it is animated and the parent having to ask) is probably the right path.

If we get that landed, there is also some fun ideas about "auto blitting" where based on what artists report them selves as stale, we can keep a cache of snapshots as we render and then only have to render things later in the order than the first stale artist. However that is moonshot ideas.

In the short term we should do what ever semi-hacky thing we need to unbreak MNE .

[timhoffm]

In the short term we should do what ever semi-hacky thing we need to unbreak MNE .

To be checked: #30591 may already unbreak the test, because it handles "vanishing" background images due to canvas swapping. However, I'm not clear whether that leads to an outdated image and we'd need to do a full redraw instead. Would it make sense to merge #30591 as is and move from there? It's not the full solution to #30503 but IMHO a reasonable change in itself.