feat: pixel ratio

[fand]

This PR adds support for high pixel ratio rendering.
It allows us to get crisper output in HiDPI environments like Retina MacBook or iPhone.

Changes

• Add Terminal::set_pixel_ratio(), Renderer::set_pixel_ratio()
  • In wasm, BeamtermRenderer.setPixelRatio() is available
• Set canvas size and styles considering the pixel ratio in Renderer::resize()

Screenshot

I edited wave_effect example (commit 2fd2ebd) to see the text and line edges quality.
Note that we need to checkout the Ratzilla PR to run the demo properly:
ratatui/ratzilla#146

Both screenshots are taken on MacBookPro (window.devicePixelRatio = 2.0).

Before

Text, line edges and cell borders look blurry.

After

Edges are crisper.
We still see artifacts around border but it's antialias, not a bug.
Maybe we can use old-school bitmap fonts if we wanna avoid the antialias...?

[junkdog]

Before
After

what am looking at? is this for 2x, 3x upscaling w/o blurring, on e.g. hires screens?

[kofany]

Hey! Great work on this PR - we really need pixel ratio support for our terminal app (terX).

I noticed that this PR doesn't include the fix for mouse selection coordinates in mouse.rs. Currently, pixel_to_cell uses event.offset_x()/offset_y() (CSS pixels) divided by cell_width/cell_height (physical pixels from atlas), which causes selection to be offset on HiDPI displays.

Here's what needs to be added to mouse.rs:

// The pixel_to_cell closure needs access to pixel_ratio
let pixel_to_cell = move |event: &web_sys::MouseEvent| -> Option<(u16, u16)> {
    let x = event.offset_x() as f32;
    let y = event.offset_y() as f32;

    // offset_x/y are in CSS pixels, cell_width/height are in physical pixels
    // Convert cell dimensions to CSS pixels for correct coordinate mapping
    let css_cell_width = cell_width as f32 / pixel_ratio;
    let css_cell_height = cell_height as f32 / pixel_ratio;

    let col = (x / css_cell_width).floor() as u16;
    let row = (y / css_cell_height).floor() as u16;

    let (max_cols, max_rows) = *dimensions_ref.borrow();
    if col < max_cols && row < max_rows { Some((col, row)) } else { None }
};

Since we need this for our development, I'll implement the full fix (including dynamic DPR tracking for window moves between displays) in my fork. Happy to share the implementation once it's working!

[fand]

Hi folks, Thanks for the comments!!
I was testing this on Ratzilla PR first (ratatui/ratzilla#146) to see if it'll work in my private project.

@junkdog
Sorry for leaving te PR without explanation.... 🙇
I edited the wave_effect demo temporarily to see text & line rendering quality.
I took the screenshots on my MBP while changing the browser zoom, then cropped the center area.

@kofany
Ah that's true! I'm gonna pick your change and try it tomorrow, thx.
BTW I was rebasing my commits locally and force-pushed today 😇 It might cause merge conflict on your side, sorry about that..

[junkdog]

Sorry for leaving te PR without explanation.... 🙇

i apologize if i came off as dismissive or short, i didn't mean it that way - but i see how it could be read as such. my bad.

Maybe we can use old-school bitmap fonts if we wanna avoid the antialias...?

https://github.com/be5invis/Iosevka looks pretty close to a pure terminal font, depending on resolution. the "problem" with bitmap fonts is that emoji still have to be rasterized from a ttf/otf, so it entails a bit more work to get it going, and would likely be processed into a .atlas file.

nice work on the PR, i took a quick look - i'll delve deeper later today or tomorrow.

a couple of questions:

• does the current impl distinguish between dynamic and static font atlases? for the dynamic atlas, i'm thinking it might be easier to just upscale the font (feat(atlas): runtime font atlas replacement #73 enables this via Terminal::replace_with_(static|dynamic)_atlas) rather than upscaling. we could document it ofc, but people are generally pretty bad at reading documentation - even pre-LLM.

• non-integer pixel ratios will have a lot of scaling artifacts. i don't know if pixel ratio can ever be 3.0, but maybe set_pixel_ratio() should take a plain enum with 2 (or 3) variants. what are your thoughts?

[fand]

Hi @junkdog !
I saw your PR, it's awesome that we can change the font size at runtime!
I wasn't aware of the problem with emojis... Thank you for clarification.

Anyway, here are my answers to your comment.
TL;DR: I still think pixel ratio support is reasonable.

Upscaling the font

for the dynamic atlas, i'm thinking it might be easier to just upscale the font

In this PR there's no distinction between dynamic/static atlas.
However, unfortunately, using large font atlas doesn't solve the problem on HiDPI displays.

In WebGL, the output is always calculated based on the logical canvas size (canvas.width and canvas.height), which can be smaller than the actual pixel size in HiDPI devices.
The browser renders the GL scene in lower resolution than the physical resolution on the screen, regardless of the internal calculation, texture resolution etc.

I made a visualization to explain why we need to scale the viewport using device pixel ratio.
https://amagi.dev/device-pixel-ratio-visualizer/

Say we have a font atlas baked with font size = 128px, cell size = 16px, and devicePixelRatio = 2.0.

When rendered without considering pixel ratio,

1. the browser renders the letter in 16x16, then
2. scales it to 32x32 (physical size on the screen).

This degrades quality at both steps. We can disable the bilinear filtering on the 2nd step with image-rendering: pixelated; but it's still not the best output.

On the other hand, when rendered with pixel ratio, the browser renders the letter in the physical resolution, resulting in a better output quality.
There's still a jaggy at the edge in the 2nd output, but we have to change the texture filter to solve this.
I think it's out of scope of this PR.

Non-integer pixel ratio

non-integer pixel ratios will have a lot of scaling artifacts. i don't know if pixel ratio can ever be 3.0, but maybe set_pixel_ratio() should take a plain enum with 2 (or 3) variants. what are your thoughts?

I'd say we should accept non-integer pixel ratio.

In the current market, there are a lot of devices with non-integer pixel ratio screens.
My main smartphone (Nothing Phone 3(a)) have 2.625.
You can see the list of dPR on recent devices here:
https://viewportsizer.com/devices/

Also, window.devicePixelRatio changes when you zoom in/out your browser window (with Ctrl++/-).
You can see your pixel ratio changing to non-integer value while zooming on this page:
https://johankj.github.io/devicePixelRatio/

Let me know if there's any concerns 🙏

[fand]

I'd mention that many other WebGL based libraries support window.devicePixelRatio.

• Three.js
  • docs https://threejs.org/docs/#WebGLRenderer.setPixelRatio
  • code https://github.com/mrdoob/three.js/blob/3fddc5cec7810814347626097623a1349ac656d9/src/renderers/WebGLRenderer.js#L621-L682
• Babylon.js
  • docs https://doc.babylonjs.com/typedoc/interfaces/BABYLON.EngineOptions#adapttodeviceratio
  • code https://github.com/BabylonJS/Babylon.js/blob/e0e8027225fd2bc87c802481996591f0e4cef7f6/packages/dev/core/src/Engines/abstractEngine.ts#L2104-L2126

[fand]

@kofany
I noticed that the mouse pos calculation is working properly in this PR now.
Since the pixel ratio only affects the final viewport res, the cell size and mouse pos are always calculated in logical size (CSS pixel).
I confirmed it's working in the canvas_waves demo:

mouse_trim.mp4

Maybe it was a problem in the commits I overwritten with force-push..
Anyway, thx for the feedback!

[junkdog]

However, unfortunately, using large font atlas doesn't solve the problem on HiDPI displays.

this can go in a different PR, but i was thinking that the dynamic font atlas could simply scale the font_size. the offscreen canvas and target canvas would need to have the pixel ratio overridden to 1.0 to render at native resolution.

I made a visualization to explain why we need to scale the viewport using device pixel ratio.
https://amagi.dev/device-pixel-ratio-visualizer/

but here you're downscaling from a 128x128 to a smaller target resolution; let's say the pixel ratio on some device is 1.26: a 10x16 glyph would be upscaled and rendered at 13x20, with some pretty nasty artifacts (this is true for the existing blurry version too ofc, but less noticeable)

In the current market, there are a lot of devices with non-integer pixel ratio screens.
My main smartphone (Nothing Phone 3(a)) have 2.625.

window.devicePixelRatio changes when you zoom in/out your browser window

since upscaling from lowres at non-integer scales produces artifacts (e.g. 10x16 to 13x20), my thinking was that we could maybe override the the canvas' pixel ratio to the nearest integer value. this would slightly change the terminal size but it would still look good when upscaling (although, it would snap between zoom levels).

sorry for taking so long to review this - i'm pretty swamped this week, but i'll do a proper review on saturday, and then the plan is to do a new release once this PR is merged.

[junkdog]

hi, thanks for the PR, impl looks good and addresses a real issue.

however, i'm hesitant about exposing raw pixel_ratio directly to users. for beamterm, this feels like an impl detail that should be managed internally. as it stands, users would need to:

• set up boilerplate to track devicePixelRatio changes (browser zoom, window moving between displays)
• have a grasp of which values produce good results (integer vs fractional scaling artifacts with static atlases)
• understand the differences between static and dynamic atlases

i'd prefer a policy-based api where beamterm handles the complexity, something like:

enum PixelScaling { // (for example)
    Off,  // resizes to native resolution
    Auto, // static: round to nears int, dynamic: scale font_size
    Fixed(NonZero<u8>), // 1x, 2x, 3x
}

beamterm would internally track devicePixelRatio changes (regardless of policy); the enum controls how each atlas type responds. PixelScaling::Auto would be the default (and most similar to how it works today, except w/o the blur)

i realize this is a larger scope than what you signed up for. so going forward, feel free to chime in other suggestions, but as i see it, our options are:

• if you're interested, you're more than welcome to continue working on the PR
• i can take over and build on your work (i'll credit you ofc)
• we merge a minimal version with just internal Auto scaling

how do you want to proceed?

[junkdog]

this function does not exist

[junkdog]

this calls Renderer::resize but the terminal grid isn't resized (ref Terminal::resize)

[junkdog]

how do you want to proceed?

have you given it any thought?

[fand]

Hi @junkdog ! Sorry for the late response... I didn't have time to work on this PR last week.
It looks like you've already shipped the fixed version? Awesome! 🎉

however, i'm hesitant about exposing raw pixel_ratio directly to users.
for beamterm, this feels like an impl detail that should be managed internally.

Yeah, that totally makes sense.
I initially chose to expose the low-level API just because it's common in WebGL libraries.
Managing the pixel ratio automatically looks more ergonomic, I love it.

integer vs fractional scaling artifacts

For this problem, I was leaning to fractional scaling because:

• Using interger scaling, the pixelBuffer-to-screen scaling artifact will persist
• Fractional scaling has artifacts too, but there's more room for workaround (e.g. snap vertices to pixels in shader)

That said, that's a trade-off. And in beamterm, sticking to integer pixel ratio looks quite reasonable, given its cell-based rendering architecture.

I tried the JS demo on the latest version and it's working perfectly!
Thank you for your work 🙏