DPI handling and multiple screens

[ocornut]

I've been toying with DPI handling recently. Part of this is motivated by the work on Multiple Viewport #1542 which are making this topic trickier than it already was before. Some (early and experimental) commits are being pushed to the Viewport branch.

This is mostly a thread to link commits too, write down some notes and eventually attract feedback, suggestion or solutions.

The work involve

• imgui-side changes (how does it affects fonts? style? various sizes and positions)
• platform-side work

Single viewport DPI handling can be handled by the application at the moment (by using a larger font and scaling e.g. the style appropriately), however we ought to clarify and simplify the work that needs to be done, and support it in the examples. The position/size of windows relative to their viewport also ideally ought to be scaled.

Multi viewport DPI handling involve changing those depending on where we are located: moving an imgui window from screen 1 to screen 2 can affect sizing. Effectively calling Begin() can put you in a different DPI context. It's trickier to solve because e.g. our font system currently don't allow for arbitrary quality scaling so we may have to come up with scheme to make that more transparent eventually. Likewise for style (and scaling style sizes is a lossy operation).

Platform-side: At the moment I am focusing on Windows. I imagine (hope) MacOS will be easier but I don't have reliable access to a Mac right now. I don't know about Linux. The platform dependent part of the work is not very complex. Under Windows, consideration of backward compatibility with older versions of Windows and Windows SDK makes the code more messy, but nothing really hard.

[G-glop]

Maybe SDF fonts? Rendering is simple/fast, they scale to any DPI, and you get prety things like outlines or shadows, and subpixel rendering for free.

[jdumas]

One also need to distinguish between Retina displays (on macOS), which are also hi-dpi screens, and regular hi-dpi external screens. On a Retina display on macOS there is a pixel ratio of 2.0 between the size of the FrameBuffer, and the size of the application window. This means that to get shap fonts, at least with the current settings, you need to load the font at 2x the size, and then set ImGui::GetIO().FontGlobalScale = 1.0 / pixel_ratio;.

And then there is the hi-dpi scaling factor (Xft.dpi on Linux, usually set in a file called ~/.Xresources) for regular screens. To get proper text size that respects this user setting, I need to render the fonts with a size of font_size * hidpi_scaling. This implies that all windows and menu sizes needs to be scaled by a factor of hidpi_scaling * io.FontGlobalScale to end up being more or less the same the layout.

At least that's how I implemented it when I dabbled into this for libigl. I ended up checking the hidpi_scaling before rendering every frame, reloading the fonts and setting the io.FontGlobalScale if necessary,to accommodate for situations where a window is moved between screens with different DPI.

Note that screens with different DPI is possible on macOS natively, but is only possible on Linux with Wayland. X only support a global DPI setting.

On the backend side, for GLFW, I get the pixel ratio as glfwGetFramebufferSize()/glfwGetWindowSize(), and the hidpi_scaling factor is given by glfwGetWindowContentScale() (function only available in the upcoming version 3.3). You might want to check how they implement this function if you want info on the backend side of things.

This approach I described is a bit annoying because I make all my menu size be proportional to the hidpi_scaling factor. At the time I couldn't figure out a better way to apply a global zoom to ImGui that also affects padding, etc. Maybe that's something to think about.

And +1 for SDF fonts, outlines and arbitrary zoom would be pretty cool things to have.

Hope this helps.

[hateom]

Is there a reliable way to simulate it for testing purposes? I don’t have HiDPI/Retina display and using some software emulators didn’t work for me.

[hateom]

OK, found a way to simulate it on macOS here: https://gist.github.com/simX/3191869

[ocornut]

If you are interested in DPI handling, imgui_impl_win32.cpp in the Viewport branch include a few extra helpers:

// DPI-related helpers (which run and compile without requiring 8.1 or 10, neither Windows version, neither associated SDK)
void ImGui_ImplWin32_EnableDpiAwareness();
float ImGui_ImplWin32_GetDpiScaleForHwnd(void* hwnd);       // HWND hwnd
float ImGui_ImplWin32_GetDpiScaleForMonitor(void* monitor); // HMONITOR monitor
float ImGui_ImplWin32_GetDpiScaleForRect(int x1, int y1, int x2, int y2);

The particularity of those helpers if they they compile without the latest 10-ish Windows SDK and they don't require the EXE to link with Windows 10-era DLL. They manually load functions from Windows DLL and redefine flags/enums locally, which appears to be standard way to achieve both. Most big software would do the dynamic loading fulfill the "can run on older Windows" requirement, but few projects needs to care about "can build with older SDK".
Just pointing out that those functions exists, and I imagine they can be merged in master earlier if needed.

[bhack]

Any news on this?

[bryanwagner]

I wanted to leave a note here on an approach I just took to handle high DPI at any resolution for a game engine. I'm so impressed by how easy it was to override data in ImGui!

I don't need high DPI resolution, just a reasonable upscale. I'm doing post-processing effects on the frame anyway, so my approach is to: 1) render scene 2) render ImGui widgets to texture via FBO. My FBO size is fixed, and much smaller than high DPI resolution.

The only change I needed to make was after calling ImGui_ImplSDL2_NewFrame and before calling ImGui::NewFrame(). I replace values for ImGuiIO fields DisplaySize, DisplayFramebufferScale, and MousePos. The first two were easy; just set them to the FBO w/h and set the scale to (1, 1). To correct the mouse position, I pulled the source code from the SDL2 implementation, did the same calculations, and scaled at the end:

float scaledMouseX = mx * (fboW / static_cast<float>(windowW));
float scaledMouseY = my * (fboH / static_cast<float>(windowH));
io.MousePos = ImVec2(scaledMouseX, scaledMouseY);

I was just really excited that it was so easy to do what I wanted and wanted to share. I scale my window size based on the desktop resolution (for windowed mode I hack 96% of the desktop height to account for OS titlebar/chrome) and render the FBO to screen. My ImGui widgets (and fonts) are now normalized independently of the resolution, and look the same on my Win8 desktop and Win10 laptop. The FBO render is relatively cheap, so I also get a significant performance boost by rendering to texture.

[mosra]

Here's how I'm implementing HiDPI support for the ImGui integration in Magnum with latest ImGui (1.66b). Hopefully this info will be useful to somebody :)

In order to seamlessly account for how HiDPI is done on Linux, Android, macOS/iOS and Windows, I'm exposing these three parameters (all of them two-dimensional float vectors):

• uiSize, to which the widgets and windows are positioned
• windowSize, to which all input events are related
• framebufferSize, amount of real pixels of the resulting window

On Windows, Linux and Android, windowSize is always the same as framebufferSize and the uiSize changes in inverse relation to the desktop scaling in the system-wide settings (which is Xft.dpi on Linux and the slider in Windows Control Panel). On macOS/iOS, windowSize is different from framebufferSize (and the uiSize is usually equal to windowSize). As a side effect of exposing these three parameters, by changing just the uiSize the user can easily apply arbitrary scaling independently of pixel density.

Then, on the ImGui side:

• Font sizes are multiplied on load by framebufferSize/uiSize ratio. Then I'm setting FontGlobalScale to uiSize/framebufferSize so the font glyph cache gets supersampled, but its size relative to the UI stays the same.
• For rendering, the projection matrix matches uiSize and besides that I'm multiplying scissor rectangle coordinates by framebufferSize/uiSize. Rasterizer takes care of the rest.
• And finally, I'm multiplying all MousePos by uiSize/windowSize, because ImGui expects that events use the same coordinate system as widget positions.

The following web demo is HiDPI-enabled and it should show everything crisp and in the same physical size on both normal and Retina screens: https://magnum.graphics/showcase/imgui/

[OvermindDL1]

@mosra How do you handle when windows are dragged between two monitors with different DPI? The traditional method is to keep rendering at the highest of the needed DPI then downsample on the other monitor only switching to the lower DPI render once the window fully exists on the lower-DPI monitor. This is a case often missed by most things though and it can make sizing of things a bit odd otherwise. ^.^;

[mosra]

@OvermindDL1 I'm able to respond to DPI change events and adjust the three above parameters based on that. All with the assumption that DPI change event (WM_DPICHANGED on Windows) is sent either when a window is partially on a higher-DPI monitor (so it needs to supersample) or when it's fully dragged to a lower-DPI monitor (so it can downsample again).

[newincpp]

I'm currently implementing Imgui on my glfw/vulkan engine. I used the imgui_impl_vulkan.cpp and imgui_impl_vulkan.h to ease the integration unfortunately I got a DPI problem (imgui used only 1/4 of the screen and the mouse acted like it was fullscreen).
I found a fix that work for me but It feel kinda hacky so I'm not sure I'm gonna make it a pull request.
You can take a look at this commit: https://git.psychoscientists.com/newin/LunaticPlatypusV2/commit/44d7b4988ef320c7977125998140b598d4fa8b01 and more precisely imgui/imgui_impl_vulkan.cpp
(the clear value change isn't important, that's just me trying to understand stuff)
Point is:
I added:

ImGuiIO& io = ImGui::GetIO();
float fb_height = io.DisplaySize.y * io.DisplayFramebufferScale.y;
float fb_width = io.DisplaySize.x * io.DisplayFramebufferScale.x;
draw_data->ScaleClipRects(io.DisplayFramebufferScale);

it fixed most of my scissor related problems
and changed those 2 lines:

viewport.width = draw_data->DisplaySize.x * io.DisplayFramebufferScale.x;
viewport.height = draw_data->DisplaySize.y * io.DisplayFramebufferScale.y;

Which fixed the rest
i'm very new with imgui so I have no idea if I'm doing it wrong or not. If that's good enough maybe I can generate a patch or a PR depending on what you prefer

[ocornut]

Thank you @newincpp . I assume you are running on Mac OSX with a Retina screen? (Afaik DisplayFramebufferScale is only > 1 on OSX).

I have now applied to the Vulkan back-end similar changes made recently to Metal/GL2/GL3 renderer to handle this via the ImDrawData::FramebufferScale field. This is written differently but should match the logic of your changes.

[Rob2309]

Hey, I am currently trying to implement a rust backend for ImGui. Properly handling DpiScale seems to be a huge challenge for me and I could not find a better place to ask this. Sorry if this is the wrong place.

The problem I have is that there seems to currently be no way to handle multiple monitors with different DPI. On windows, cursor and window positions/sizes are all in physical coordinates, which I can pass to ImGui without a problem, however, incorporating the necessary UI "zoom" factor is something I don't know how to do. Setting the framebuffer scale is not appropriate since it is a global setting instead of per viewport. Setting viewport.DpiScale seems to have no effect for me (hope I am not doing anything wrong on this part).

In the FAQ, I can see that there currently are workarounds for handling the font dpi properly, but there seems to be no discussion on how to properly scale the UI as a whole, which would be necessary on high-dpi monitors to not make everything too small to read.