Complex text layout, in particular with TeX input [was: MathJax does not support Complex text layout.]

[hartman]

Because MathJax looks at individual code points it has trouble dealing with scripts that require bidirectionality, context shaping etc. This is visible whenever trying to use hebrew or arabic for instance.

It would be good if MathJax would be able to identify these ranges and be able to keep those as blocks instead of dividing it into individual characters. At the very least in \text mode.

http://en.wikipedia.org/wiki/Complex_text_layout

[dpvc]

Note that if you set mtextFontInherit to true in the HTML-CSS and SVG sections of your configuration, then MathJax will process \text{} as a single <span>, and so that should do as you request. You are right that MathJax could do better when mtextFontInherit is false. It should group "unknown" characters into a single collection, rather than putting each into a separate <span>.

[dpvc]

PS, I saw the report on the Wikimedia bugzilla and was planning to add it to the list of things to fix. Thanks for staring the issue here to track that.

[hartman]

Thanks for the mtextFontInherit tip. I was going to enable that anyways, but this is one more reason to do that.

[dpvc]

Some support for RTL was added in v2.3, but the issue of multiple-character sequences being treated as a unit remains. For \text{}, these characters should already be grouped into a single <span>, so that would be one way to handle it, though not very convenient.

Ideally, MathJax would put each sequence that forms one group into a single <mi> or <mo>, just as it does for single Latin letters now. I've looked into this to some degree, and there are some difficulties handling it. It is possible to have combining characters grouped with their preceding characters, but it is not clear to me how some characters work. For example, it seems that the virama (U+0D4D) combines not just the character on its left, but also on the right, though I might be misunderstanding it. It also seems that some of these grouping are handled by ligatures within the fonts, not by combining characters. Unfortunately, MathJax does not have access to ligature information from the fonts. While it would be possible to add ligature data to MathJax's font tables, this could be a significant amount of data very little of which would be used by any one page.

I'm really not familiar enough with the languages that use these features to know if what I'm trying out would be sufficient or not. I'm wondering if it is possible to get some examples from a variety of languages that show the range of situations that need to be accommodated.

One approach might be to put the data needed for each language's script into an individual extension that gets loaded for those pages that need it (either explicitly in the MathJax configuration, or via \require{} within the math on the page). Do you think that would be acceptable?

[hartman]

Perhaps @amire80 of our WMF language engineering is able to help out a bit here...

[pkra]

@hartman do you think you could poke @amire80 some time? We'd love to improve this, especially if Wikipedia wants to roll out the SVG output more widely.

[amire80]

I'm right here :)

How can I help?

Testing? - Gladly, just tel me what to test exactly.

Examples of how non-Latin scripts work in formulas? - It's not used in Hebrew textbooks, but it is used in textbooks in Arabic and Persian. Maybe @ebraminio can chime in here.

Anything else?