Handling mouse events

[audetto]

I am using 1.84 WIP.

I would like to handle mouse events and following instructions I do not handle them if ImGui wants to (io.WantCaptureMouse).

Basically I would like to know for a particular ImGui window

1. When it is selected
2. In which case, were the mouse is relative to its position / size.

My problem is that as soon as the mouse is over a window, I get io.WantCaptureMouse and so I loose the event.

Otherwise I guess I could use functions in this area

imgui/imgui.h

Lines 336 to 344 in 81eceb0

	IMGUI_API bool IsWindowAppearing();
	IMGUI_API bool IsWindowCollapsed();
	IMGUI_API bool IsWindowFocused(ImGuiFocusedFlags flags=0); // is current window focused? or its root/child, depending on flags. see flags for options.
	IMGUI_API bool IsWindowHovered(ImGuiHoveredFlags flags=0); // is current window hovered (and typically: not blocked by a popup/modal)? see flags for options. NB: If you are trying to check whether your mouse should be dispatched to imgui or to your app, you should use the 'io.WantCaptureMouse' boolean for that! Please read the FAQ!
	IMGUI_API ImDrawList* GetWindowDrawList(); // get draw list associated to the current window, to append your own drawing primitives
	IMGUI_API ImVec2 GetWindowPos(); // get current window position in screen space (useful if you want to do your own drawing via the DrawList API)
	IMGUI_API ImVec2 GetWindowSize(); // get current window size
	IMGUI_API float GetWindowWidth(); // get current window width (shortcut for GetWindowSize().x)
	IMGUI_API float GetWindowHeight(); // get current window height (shortcut for GetWindowSize().y)

is this correct?

[PathogenDavid]

It's really hard to tell exactly what you're trying to accomplish based on your description. It'd be helpful to know what your end goal is.

WantCaptureMouse being set when hovering windows is intentional. It sounds like you're trying to perform a UI interaction though, so I'm not sure why you're checking it. (The intent behind WantCaptureMouse is to prevent your game from reacting to mouse events which might also be handled by Dear ImGui.)

[audetto]

Fully agree with your statement.

So, in the picture the green bit is the video of the emulator.
There is a window entirely filled with an image.

I would like to receive mouse events when the mouse pointer is over the image (the green bit) (ideally in x,y relative to the image).

In most GUI frameworks, every widget has a virtual mouseEvent, which is called when the mouse is directly over the component, how do I do it here?

[PathogenDavid]

Ah that makes much more sense, thanks for clarifying!

One of the biggest differences between Dear ImGui and many other UI frameworks is it is not inherently event-driven.

Assuming you're submitting your screen as an Image widget, I would recommend using a combination of IsItemHovered, GetMousePose, and GetItemRectMin to get what you want. You likely also want IsItemFocused for keyboard and mouse button handling.

Here's a quick example:

// Using the font texture as an example, you'd obviously use your emulator's screen texture instead
ImGuiIO& io = ImGui::GetIO();
ImTextureID textureId = io.Fonts->TexID;
ImVec2 textureSize = ImVec2(io.Fonts->TexWidth, io.Fonts->TexHeight);

ImGui::Begin("GH-4233 Example");
ImGui::ImageButton(textureId, textureSize);

bool isHovered = ImGui::IsItemHovered();
bool isFocused = ImGui::IsItemFocused();
ImVec2 mousePositionAbsolute = ImGui::GetMousePos();
ImVec2 screenPositionAbsolute = ImGui::GetItemRectMin();
ImVec2 mousePositionRelative = ImVec2(mousePositionAbsolute.x - screenPositionAbsolute.x, mousePositionAbsolute.y - screenPositionAbsolute.y);
ImGui::Text("Is mouse over screen? %s", isHovered ? "Yes" : "No");
ImGui::Text("Is screen focused? %s", isFocused ? "Yes" : "No");
ImGui::Text("Position: %f, %f", mousePositionRelative.x, mousePositionRelative.y);
ImGui::Text("Mouse clicked: %s", ImGui::IsMouseDown(ImGuiMouseButton_Left) ? "Yes" : "No");

ImGui::End();

Note the use of ImageButton instead of Image. Your screen is only sort-of a button, but using ImageButton allows IsItemFocused to work and avoids having the window move when you drag the mouse inside the emulator.

---

A more built-out version for your Apple ][ emulator might look something like this pseudocode:

ImGui::Begin("Apple ][");
ImGui::ImageButton(screenTexture, screenTextureSize);

if (ImGui::IsItemHovered())
{
    ImVec2 mousePositionAbsolute = ImGui::GetMousePos();
    ImVec2 screenPositionAbsolute = ImGui::GetItemRectMin();
    ImVec2 mousePositionRelative = ImVec2(mousePositionAbsolute.x - screenPositionAbsolute.x, mousePositionAbsolute.y - screenPositionAbsolute.y);

    SendMousePositionToEmulator(mousePositionRelative);
}

// If the emulator has focus, send it mouse button/keyboard events
if (ImGui::IsItemFocused())
{
    SetEmulatorLeftMouseDownState(ImGui::IsMouseDown(ImGuiMouseButton_Left));
    // ...etc for other mouse buttons

    for (int i = 0; i < IM_ARRAYSIZE(io.KeysDown); i++)
    {
        SetEmulatorKeyState(i, io.KeysDown[i]);
    }
}
// When the emulator looses focus, release all buttons
else
{
    SetEmulatorLeftMouseDownState(false);
    // ...etc for other mouse buttons

    for (int i = 0; i < IM_ARRAYSIZE(io.KeysDown); i++)
    {
        SetEmulatorKeyState(i, false);
    }
}

ImGui::End();

Note that the index for the io.KeysDown array depends on your platform backend.

---

If you need to have this be event-driven for some reason, you'd have to modify whatever platform backend you're using or capture the inputs some other way. (You'd probably still want to use IsItemHovered/IsItemFocused to determine whether to send those events through to the emulator though.)

[audetto]

I see.

At the moment this is all event driven because I display the emulator window as the background of the ImGui application, so I get (residual) events from SDL when no other ImGui widget is focused.

I see what you mean, is that I can ask any time what the mouse and keyboard state is, but I need to do some book-keeping to decide if an event has been generated or not. I guess I need to compute the delta and fire when it changes.

So in a sense I need to decide if I stick to SDL events or I jump completely to ImGui state-polling.

[PathogenDavid]

Yup, sounds like you got it! I'd probably go with whatever fits more naturally with your emulator, or just stick to SDL events since it sounds like you already have them working.

If your goal is to be able to switch between having the app as a fullscreen background or as its own window without having the logic diverge, you might check out the fullscreen window example under the Examples menu in the demo and use that for your fullscreen background mode. (The example doesn't use this, but in your case you'd want to specify the ImGuiWindowFlags_NoBringToFrontOnFocus and ImGuiWindowFlags_NoNavFocus flags to put the fullscreen window in the background.)

Or if your goal is to be able to view the debugger windows without having them on top of your emulator's screen, you might check out the experimental docking branch. You don't need to use the docking features, but an added bonus of this branch is that it lets you pull windows outside of the main viewport.

[audetto]

If your goal is to be able to switch between having the app as a fullscreen background or as its own window without having the logic diverge, you might check out the fullscreen window example under the Examples menu in the demo and use that for your fullscreen background mode.

This is exactly what I was doing.
Normally full-windowed, but occasionally in its own window to split from the other windows.
Currently works, but a bit counter-intuitive as I can only interact when it is not selected.

I will have a look as you suggest and weigh the pros and cons.

Thanks everybody.

[ocornut]

I reckon we could probably design and add an example in the demo that would showcase the use of ImageButton()/InvisibleButton() and then reacting on events.

Technically this Custom Rendering>Canvas section already does it

But something a bit more explicit may be good.

In the past I also had fun designing a demo where user could click a "game viewport" for it to fully take on inputs until ESC is pressed, but it would requires a backend agnostic way to request mouse disable. Right now that (uncommitted) demo hide the mouse cursor but mouse can still be used to click elsewhere in the OS so it's not really useful as-is:

[ocornut]

Closing this as answered, thanks for the thoughtful answers!