SDL2 backend multi-viewport window dragging doesn't track fast mouse movements

[seanmiddleditch]

Version/Branch of Dear ImGui:

Docking branch commit 1e14cc5 (very latest at time of writing), no local code changes

Version: 1.87 (18701)
Branch: docking

Back-end/Renderer/Compiler/OS

Back-ends: imgui_impl_sdl.cpp
Compiler: cl 19.31.31103 (VS 2022, 17.1.0 Preview 3.0)
Operating System: Windows 11

My Issue:

Using the sdl2 backend on the docking branch (e.g. with either example_sdl_directx11 or example_sdl_opengl3 projects):

Grab a window titlebar and start dragging/moving it.

Move the mouse fast enough so that it leaves the bounds of the window. This is easiest to do by quickly snapping the mouse upward if you're dragging by the titlebar.

The window ceases being dragging/moved after the mouse leaves the window bounds.

This appears to repro for only the sdl examples, not the win32 examples; i.e. I don't see this issue with example_win32_directx11.

Screenshots/Video

Standalone, minimal, complete and verifiable example:

Build and run the example_sdl_directx11 example, the issue repros out of the box.

[seanmiddleditch]

Ignore the below - I'm seeing it repro even on 0647ba3, it just seems... more difficult to repro? Might be entirely subjective.

Testing my project (requires docking branch), the issue appears to have begun sometime between these two merges from master:

8567a4cc <-- broken

0647ba3 <-- correct

Looks like the "meat" of the input queue and mouse events landed in the docking branch from that commit range, so that's probably not terribly surprising. :)

[ocornut]

I could not repro. I am on a source distrib of SDL ~2.0.17 for Windows.
Which SDL version are you on?
Might be worth updating if you are on a old version, just in case.

Otherwise I guess we would need to find what's interfering for you :/
To investigate further:

• Open Demo>Inputs>Mouse State and try to find out what's wrong from dear imgui's point of view: it is the mouse position that is clamped? mouse buttons that are released?
• In Metrics->Internal State how does the MouseViewport values behave when that happen?

[seanmiddleditch]

I'm using the prebuilt 2.0.16 package from libsdl.org. I'll try an upgrade and get the extra diag info posted here soon(-ish).

[JustQu]

I have the same problem. Also resizing on top of another viewport causes twitching

.

[ocornut]

@JustQu Does it also happens for you on the stock unmodified example?

[JustQu]

@ocornut yes, just builded sdl_opengl3 example and still the behavior is the same.

[ocornut]

Does 1.86 give you same result ?

[JustQu]

1.86 works fine, window moving and resizing

[seanmiddleditch]

(see comment below: maybe not actually fixed by upgrading after all)

I verified that updating to SDL 2.0.20 (using the prebuilt VC downloads at http://libsdl.org/download-2.0.php) resolves the issue for me, both in my app and in the imgui example binaries.

Happy to just call this an SDL bug and close this issue on my end (assuming @JustQu's issue is also resolved by updating).

[ocornut]

Thanks, that's good to know.

This is not ideal/great I'll still probably want to investigate a little bit with SDL 2.0.16 if we can find a workaround to minimize the issue, and at minimum we'll add commentas about it.

[seanmiddleditch]

Yeah, also, maybe hold up on that confirmation. I can't get it to repro in the example app, and the repro steps in my app are different but I can still get it to happen. Trying to figure out exactly what the repro steps are, then seeing if I can reproduce in the example app. Either way I'll try to get that diagnostic information you asked for.

[seanmiddleditch]

Short version of what I'm seeing now is that dragging docked windows out of the "main window" does not exhibit the behavior.

However, if I grab an undocked windows (with its own viewport) it does exhibit the behavior. Same if I grab a docked window out of a non-main viewport: the issue manifests. For example, with the demo window (which does not begin docked) I always exhibit the behavior, in my app.

Those repro steps are being very consistent for me in my app. Haven't been able to reproduce in example app yet, though. I'm using the stock imgui SDL and directx backends without any (intentional) trickery, but I am setting some non-default flags and window classes, so it may be influenced by one of those. I'll try to narrow it down or find the right steps to repro in example app.

[ocornut]

There are various factors involved:

The SDL backend now enable io.BackendFlags |= ImGuiBackendFlags_HasMouseHoveredViewport
As I displayed the value of io.MouseHoveredViewport:

ImGui::Text("MouseHoveredViewport: 0x%08X", io.MouseHoveredViewport);`) 

it became clear that SDL Backend reports a "flickering" value when moving near the edge of a window. I am pretty sure that commenting the line setting ImGuiBackendFlags_HasMouseHoveredViewport in the backend would essentially "fix" the issue by reverting to the old behavior of letting dear imgui guess the hovered viewport based on its knowledge of where viewports area. So we can probably revert to it.

When dragging a viewport it is temporarily set to _NoInputs for the purpose of retrieving the window "under" (for e.g. docking).
Since the SDL backend doesn't honor this flag, when that viewport is reported Dear ImGui automaticaly also reverts to guessing the hovered viewport.

Almost certainly, on the setup where this is broken, if you click and drag from near the edge of a viewport you are more likely to hit the bug as on a given moving frame you will more often stray outside the last viewport boundaries).

[ocornut]

@rokups seems to repro it too:

so far i could only notice extra AddMouseViewportEvent(0) event in cases where window fails to continue moving after detachment on a rapid mouse flick. it seems like mouse cursor may end up outside of viewport window for a brief moment and that stops drag operation.

I can't get AddMouseViewportEvent(0) here.

Logs would be useful to tell if that zero comes from a _LEAVE event or else.
In addition I don't understand what "stops" the dragging because once moving the MouseViewport gets locked to MovingWindow->Viewport. Can you put a break-point on the three occurrences of g.MovingWindow = NULL (other than constructor) to tell. Most likely it happens in UpdateMouseMovingWindowNewFrame(), it is because MouseDown[0] == false or because io.MousePos is invalid?

Here are a few debug lines to add in imgui_impl_sdl.cpp:

#include "imgui_internal.h"

            if (window_event == SDL_WINDOWEVENT_ENTER)
            {
                if (io.KeyCtrl)
                {
                    ImGuiViewportP* viewport = (ImGuiViewportP*)ImGui::FindViewportByPlatformHandle((void*)SDL_GetWindowFromID(event->window.windowID));
                    IMGUI_DEBUG_LOG("SDL_WINDOWEVENT_ENTER windowID=0x%08X viewport=0x%08X (%s)\n",
                        event->window.windowID, viewport ? viewport->ID : 0, viewport && viewport->Window ? viewport->Window->Name : "");
                }
                bd->MouseWindowID = event->window.windowID;
            }
            if (window_event == SDL_WINDOWEVENT_LEAVE)
            {
                if (io.KeyCtrl)
                {
                    ImGuiViewportP* viewport = (ImGuiViewportP*)ImGui::FindViewportByPlatformHandle((void*)SDL_GetWindowFromID(event->window.windowID));
                    IMGUI_DEBUG_LOG("SDL_WINDOWEVENT_LEAVE windowID=0x%08X viewport=0x%08X (%s)\n",
                        event->window.windowID, viewport ? viewport->ID : 0, viewport && viewport->Window ? viewport->Window->Name : "");
                }
                bd->MouseWindowID = 0;
                io.AddMousePosEvent(-FLT_MAX, -FLT_MAX);
            }

        if (io.KeyCtrl)
        {
            IMGUI_DEBUG_LOG("AddMouseViewportEvent viewport=0x%08X\n", mouse_viewport_id);
        }
        io.AddMouseViewportEvent(mouse_viewport_id);

[ocornut]

I think this may be the same as GLFW #4984 and same workaround 732bc20 could work.

I guess you are getting a SDL_WINDOWEVENT>SDL_WINDOWEVENT_LEAVE event without a SDL_MOUSEMOTION leaving MousePos invalid for one-frame which drops the moving.

If you can confirm that I think we can come up with a workaround.

[rokups]

732bc20 does indeed fix this issue 👍🏻

[ocornut]

Nice, you mean implementing the same for SDL ? Do you have that commit around so we can cherry-pick it ? Thanks !

[rokups]

Here: docking...rokups:rk/fix-5012
Fix is a little bit more involved. That jumpy window when dragged issue i mentioned... I had to use a global mouse coordinates in SDL_MOUSEMOTION to fix jumpyness. And only with that change i can no longer reproduce current issue.

[seanmiddleditch]

Verified that @rokups fix-5012 branch resolves all the cases I can think of to test locally. Dragging undocked windows, docking/undocking, dragging windows after undocking, etc. all are behaving correctly for me now.

[seanmiddleditch]

Also saw that he has a newer rk/fix-5012-again -- tested that, it also solves this locally for me.

[ocornut]

@seanmiddleditch
Does it works for you with this?
75d9296

This should be a rewording on Rokas's top-most commit (same logic but more explicit)
(Without the earlier commits which are related to another issue seemingly pertaining to X11)

[seanmiddleditch]

@ocornut - it does indeed

[ocornut]

Merged f337378
Thanks everyone for your help!
The other (unrelated) fix for stuttering cursor on SDL+X11 will be pushed separately.

[ocornut]

Note how #5082 revealed another issue.
Pushed 9b0c26b to fix in docking, also pushed c5f6721 in master which is == f337378 + 9b0c26b

[RT2Code]

Hello, does this bug still happens for you with the latest SDL2 version? I can't reproduce it on my end with SDL 2.0.22.

I pushed a fix some months ago about a related issue (libsdl-org/SDL#4832), and many more mouse related fixes were added to the SDL since then. I'd like to know if the SDL_CaptureMouse function is finally working as intended or if it still need to be fixed.

[Awuwunya]

This has been broken in a recent imgui docking commit again. After updating from 64b88da the bug happens again. I am using opengl2 SDL (2.0.22) as the backend. I suspect 51bbc70 broke this once again, but I can't confirm at the moment.

[Jonathan-L-Davis]

I'm using SDL version 2.0.22 prebuilt, opengl3, windows 11, mingw-64 gcc version 12.1.0 (Rev2, Built by MSYS2 project), and last pulled the imgui docking branch from Nov. 4, 2022. I've been using both my engine code, as well as the example_sdl_opengl3 main file, which appear to behave identically. I'm not sure if this is actually the same bug as previously, especially since it only happens when docking is enabled.

In my testing with this bug, I've found it only occurs when ImGuiConfigFlags_DockingEnable is set. I can only get the window to glitch if I am dragging by the title bar, which enables it to be a dock source. Additionally, when I comment out the "BeginDockableDragDropSource" function call in the "Begin" function, that also stops the bug, but disables docking. This does not happen when I comment out the function "BeginDockableDragDropTarget" a little later in the "Begin" function, both around line 7350.

Lastly when the mouse is dragging the window but the window is not following it (when the bug is active), the window local position does not change, but the global ones do, using the functions 'SDL_GetMouseState' and 'SDL_GetGlobalMouseState' respectively. Viewport local coordinates stay at their last values before mouse left the viewport. Anyone have any thoughts for where to look in the code? I'm running out of ideas.

[ocornut]

Thank you both. I can confirm the issue and can confirm that 51bbc70 (#5710) broke it when we changed:

SDL_CaptureMouse(bd->MouseButtonsDown != 0 ? SDL_TRUE : SDL_FALSE);

to

SDL_CaptureMouse((bd->MouseButtonsDown != 0 && ImGui::GetDragDropPayload() == nullptr) ? SDL_TRUE : SDL_FALSE);

Removing this unfortunately means that we cannot rely on io.AddMouseViewportEvent() calls while drag and dropping as SDL doesn't gives us the correct information when capturing.

Instead of completely disabling ImGuiBackendFlags_HasMouseHoveredViewport in SDL backend I am trying a more dynamic approach with fix d7aca16