Characters with different font after fallback shaped together

[miloush]

• .NET Core Version: 7.0.100-preview.4.22252.9
• Windows version: 25151.1000.amd64fre.rs_prerelease.220625-1835
• Does the bug reproduce also in WPF for .NET Framework 4.8?: Yes

Problem description:

Punctuation, whitespace and alike are treated as base characters, forming clusters with vowel marks (instead of a dotted circle being inserted) when the marks are not in the font. The Indic engine is responsible for inserting the dotted circle, but it does not get to shape the run.

Someone should split the run for shaping if the characters are to be shaped using different fonts as a result of font fallback. My initial guess is line services are to blame.

Visual Studio is one of the affected products, the rendered glyphs depend on how the colorization and tokenization splits the runs:

but

Here is an example where the character is sometimes clustered with the preceding one (as a question mark in a box) and sometimes rendered as expected with a circle:

Minimal repro:
FallbackShaping.zip

[miloush]

Found it:

wpf/src/Microsoft.DotNet.Wpf/src/PresentationCore/MS/internal/FontFace/PhysicalFontFamily.cs

Lines 308 to 316 in 89d172d

	if (!Classification.IsCombining(originalChar))
	{
	hasBaseChar = true;
	}
	else if (hasBaseChar)
	{
	// continue to advance for combining mark with base char
	continue;
	}

I will check the desired behaviour here.

[lindexi]

@miloush I remember other developer raised similar issues, but I can't find the link now. Do you remember?

[miloush]

@lindexi thanks, I didn't find a similar issue either.

OK so the reason for the current behavior is that the shaping engine can normalize the string on a character level before handing it to the font tables, so technically you can have a decomposed character in the string but only precomposed character in the font.

Since the shaping is normally broken by script, I would suggest to verify that the base character is the same script as the combining mark, and if not, check whether the mark is present in the font.