Decouple text shaping from glyph rendering to support custom shapers/renderers

[jamesthomasgriffin]

Problem

The Rasterizer and Font class have tangled responsibilities.

Proposal

Replace them with FontMetrics and GlyphAtlas which cleanly separate font metrics and shaping from rendering. FontMetrics does not know anything about msdfs, or GPU buffers; and GlyphAtlas does not know anything about UTF8 or fonts.

The Font class could live on as a composite of these two, providing partial backwards compatibility.

FontMetrics loads the font, and performs font shaping. I.e. from a UTF8 string, it generates a list of vertices (4 for each glyph in a documented order) alongside a list of glyph ids). It should also provide curve data for individual glyphs.

GlyphAtlas should a) take the curve data for a given id and fill the atlas accordingly, and b) take lists of vertices and glyph ids and get them rendered.

So a Lua implementation of a text drawing function could look like (ignoring world placement and caching of local buffers etc):

local function drawText(pass, str, fontMetrics, glyphAtlas, wrap)
    local vertices, ids = fontMetrics:getVertices(str, wrap)
    glyphAtlas:loadGlyphs(ids, fontMetrics)  -- fontMetrics is needed to provide curves for any unseen id
    glyphAtlas:draw(pass, vertices, ids)
end

local font = newFont("my_font.ttf", 32) -- scales font height to 32
local atlas = newMSDFAtlas(64) -- glyph raster height is 64

function draw(pass)
    drawText(pass, "Hello world", font, atlas, 1000)
end

This creates an opening for users, they can generate or edit their own vertices, and they can provide their own curves to the GlyphAtlas.

Some applications

• use an alternative heavyweight font shaping library that replaces FontMetrics to get the best support for the most advanced font shaping,
• load SVGs, with curve data coming from path attributes and shaping coming from the SVG tree,
• edit the vertex buffer produced by FontMetrics to animate individual letters/words for emphasis, or change the colours to support animated linear gradients,
• add font fallbacks, or the ability to add vector icons into the existing GlyphAtlas.

There is also the option to provide multiple implementations of GlyphAtlas, one could work using MSDFs, another using SLUG (or Robin maybe?), or in the orthographic, fixed size case, a basic rasterizer. All can share the same interface.

Notes

• the interface of FontMetrics would be nearly the same as Rasterizer, but without newImage (thus requiring a name change), and with added shaping methods: getWidth, getVertices etc. So migration shouldn't be too hard, though there is the scaling ambiguity between the metric methods provided by Rasterizer and Font now they are combined. If Font lived on then it could maintain its interface.
• the interface of GlyphAtlas would be new, but quite simple, perhaps just load and draw methods.

[bjornbytes]

I'm going to have low availability over the next few days, so it'll be a little bit before I can fully digest this. Here is my first impression after reading though:

• It would be helpful to explain what pain results from the tangled-ness, in the problem section. I think I can mostly piece it together from the "applications" section though.
• It's nice how the Rasterizer <-> Font API boundary is images instead of curves, because not all fonts have curves (BMFont, Image icons, etc.). Although maybe both curves and images could be accepted.
• It seems like FontMetrics is very close to Rasterizer, and GlyphAtlas is very close to Font. Maybe this proposal could be framed as changes to those objects instead of creating new ones?
• I do definitely get the feeling that moving the "text shaping" parts out of Font/Rasterizer and into some other object (Shaper?) would be a good idea. The shaping code probably needs to own the font file, metrics, etc. instead of rasterizer.
• There could be 2 shaping backends: a basic one (what exists today) and a fancy one that uses a library.

Overall though, I agree that some untangling is needed here!

[jamesthomasgriffin]

Ofcourse, there's no rush with this, I think taking some time to get it right is important. I'll just note down some of my thought process when writing the above:

• I initially tried to keep the Rasterizer / Font classes, but my problem was that really the names are the wrong way round. Rasterizer is responsible for the font metrics, while Font is responsible for the implementation details of glyph rendering. Given that the rasterization process is tied to the rendering implementation, it should be the job of the Font class, and given that font shaping is dependent on the font metrics, it should really be the job of the Rasterizer class. But making those changes means that now the Rasterizer class doesn't rasterize and the Font class doesn't 'font'. Swapping the names resolves this, but this might lead to the most confusion, although in the long run it might be the correct choice? So that's why new names are in the proposal, they might not be the best names.

• I also toyed with the idea of creating new classes for shaping and for rasterizing.

  • For shaping I thought that whatever external library used, Harfbuzz, stbtt, kb_text_shape, it would make sense if this library handled all the font metrics as if they do shaping they do everything else! So I think adding an extra Shaper class might overcomplicate things.
  • For rasterizing, again the implementation of the glyph rendering entirely determines the method of rasterization. But you might have multiple ways of providing a msdf, either generated by the msdf dll at runtime, or in advance by msdfgen or another tool. This requires more thought.

• I didn't add this to the proposal, but I think bitmap fonts could be supported, the FontMetrics class needs to produce some data for each glyph, in the proposal I just included curves, but it could be a bitmap. The GlyphAtlas needs to take whatever data is provided, process it if needed and then cache the results. If the implementation is just going to be a standard texture atlas, then it can just copy the bitmap data in.