`MouseCallback` requires existing window to work

[MatusGasparik]

Hey Alexander,
I'm trying to use your library to work with OpenCV in a way that feels more pythonic.
I am having problems getting the MouseCallback to work. Here a simple example:

import cv2
from pcv.vidIO import VideoReader
from pcv.interact import MouseCallback

def on_mouse_click(event, x, y, flags, params=None):
    if event == cv2.EVENT_LBUTTONDOWN:
        print("Click", x, y)

window = 'foo'
with VideoReader("my_video.mp4", display=window) as vid:
    vid.stream(mouse_handler=MouseCallback(window, on_mouse_click))

But, I don't get any outputs. Is there some I should be doing differently or is this a bug?

[ES-Alexander]

Hi @MatusGasparik,

At least at first glance your code looks fine. I’ll run some tests tomorrow to see if I can figure out what’s going wrong / how to get the mouse callback functionality working as expected.

Hope the library is going well for you outside of this issue :-)

[MatusGasparik]

Thanks, @ES-Alexander for your reply. To be honest, I'm still figuring out how to use your library and at the same time learning OpenCV. I didn't feel comfortable with the use of global variables and the lack of automatic resource management of a "typical OpenCV script" and I find your approach really great.

I had problems trying to do other stuff using your library, but probably the problem was with my approach. I will try a few more things and if I won't figure it out I will ask you...

[ES-Alexander]

I'm still figuring out how to use your library and at the same time learning OpenCV. I didn't feel comfortable with the use of global variables and the lack of automatic resource management of a "typical OpenCV script" and I find your approach really great.

Fair enough - glad you appreciate the approach! Hopefully the examples in the README and examples folder are helpful with that, but I do understand this library currently lacks comprehensive documentation, so feel free to raise issues here if you run into problems, or ask for help on r/learnpython or r/opencv as appropriate :-)

---

On the actual issue, you can only set a mouse callback on a window that already exists. Basically you can fix your example by making a cv2.namedWindow(window) before starting to stream. That's something I should likely add some documentation for to make more obvious, so thanks for raising this :-)

Assuming you're using Python 3.8 or later you could do the following:

import cv2
from pcv.vidIO import VideoReader
from pcv.interact import MouseCallback

def on_mouse_click(event, x, y, flags, params=None):
    if event == cv2.EVENT_LBUTTONDOWN:
        print("Click", x, y)

# create the window so mouse callbacks work 
cv2.namedWindow(window := 'foo') # assign a 'window' variable, and create a namedWindow using its value
with VideoReader("my_video.mp4", display=window) as vid:
    vid.stream(mouse_handler=MouseCallback(vid.display, on_mouse_click))

Before Python 3.8 the walrus/assignment operator (:=) didn't exist, so you would need to do a normal variable assignment (e.g. window = foo) on its own line, and then use the window variable to create the namedWindow :-)

[ES-Alexander]

In addition to clearer documentation, it may be possible to check for this at runtime using cv2.getWindowProperty(window, cv2.WND_PROP_VISIBLE) != -1 (which could be done in MouseCallback.__enter__), but it seems like this approach only works with the Qt backend. Ordinarily that's fine for users who've installed opencv via the standard opencv-python pip install, but may be problematic for other users. Also, Mac uses the Cocoa backend by default, and is unlikely to switch back to default Qt.

Probably most effective to just

• print something like Note: mouse callbacks require an existing window to assign correctly (e.g. via cv2.namedWindow) on initialisation, and
• add a direct usage example in the README (perhaps even the one in this thread)

[MatusGasparik]

On the actual issue, you can only set a mouse callback on a window that already exists. Basically you can fix your example by making a cv2.namedWindow(window) before starting to stream. That's something I should likely add some documentation for to make more obvious, so thanks for raising this :-)

Assuming you're using Python 3.8 or later you could do the following:

import cv2
from pcv.vidIO import VideoReader
from pcv.interact import MouseCallback

def on_mouse_click(event, x, y, flags, params=None):
    if event == cv2.EVENT_LBUTTONDOWN:
        print("Click", x, y)

# create the window so mouse callbacks work 
cv2.namedWindow(window := 'foo') # assign a 'window' variable, and create a namedWindow using its value
with VideoReader("my_video.mp4", display=window) as vid:
    vid.stream(mouse_handler=MouseCallback(vid.display, on_mouse_click))

Before Python 3.8 the walrus/assignment operator (:=) didn't exist, so you would need to do a normal variable assignment (e.g. window = foo) on its own line, and then use the window variable to create the namedWindow :-)

Many thanks for the suggestion. I haven't had yet any time to look into it, but as it seems, registering the MouseCallback actually predates the creation of the cv2.namedWindow in the VideoReader (or up the inheritance stack). Would it be a simple fix to reverse the order?

[ES-Alexander]

as it seems, registering the MouseCallback actually predates the creation of the cv2.namedWindow in the VideoReader (or up the inheritance stack). Would it be a simple fix to reverse the order?

The VideoReader doesn't actually directly create a window at all - the stream functionality engages the optional MouseCallback, and then just displays each frame to the specified window name using cv2.imshow. imshow automatically creates the window if it doesn't exist, which means the stream functionality will just as happily use an existing window as create a new one, so there isn't really an order to reverse.

That being said, I've just checked the namedWindow docs to see what it does if the window already exists, and it apparently does nothing, so that can definitely just be added to the stream function before the MouseCallback is engaged, which would fix this issue :-)