Add math styling module

[mkorje]

This PR adds functions and the related infrastructure needed to take a character/string and a style in math (e.g. blackboard-bold, monospace), and return it in styled form. It is implemented under the feature styling, which I've added as a default.

Moving this to Codex was brought up briefly on Discord, and whilst it doesn't directly fall under "naming Unicode symbols", I think it still fits quite nicely in this crate. I may have jumped the gun a little on this, but it wasn't much work to do. Hopefully this PR starts some more thorough discussion of this idea. I also think that moving this outside of the main typst repo will be an improvement and make it more maintainable.

[MDLC01]

A bit more context: #5 (comment).

[MDLC01]

Regarding the implementation, as well as having many to_{style} functions, there could also be a MathStyle::apply method to apply a style to a char.

[MDLC01]

Regardless of my above comments, @laurmaedje will have to approve moving those functions before we merge this PR.

[mkorje]

Regarding the implementation, as well as having many to_{style} functions, there could also be a MathStyle::apply method to apply a style to a char.

The issue I had with something like this is that some of the styles will output multiple characters.

I also considered implementing the MathStyling trait for EcoString, as EcoString is used in the Typst repo. Though I wasn't sure about adding that as a dependency; maybe instead it could be under a feature flag?

[MDLC01]

Regarding the implementation, as well as having many to_{style} functions, there could also be a MathStyle::apply method to apply a style to a char.

The issue I had with something like this is that some of the styles will output multiple characters.

Can you not return [char; 2] like the to_style function? Essentially, making to_style a method on MathStyle.

I also considered implementing the MathStyling trait for EcoString, as EcoString is used in the Typst repo. Though I wasn't sure about adding that as a dependency; maybe instead it could be under a feature flag?

Instead, maybe to_styled could be made generic over its return type T: Default + AddAssign<&str>. Or maybe I'm just overthinking, and we can just add the dependency behind a feature flag.

[MDLC01]

Also, you can implement MathStyling for str instead, that way anything that derefs to str (such as String or EcoString) can benefit from to_styled.

[laurmaedje]

Looks like there's quite a bit more going on here than in typst-layout/src/math/text.rs, e.g. the looped and chancery stuff. Would that somehow be exposed in Typst or is it for completeness?

[laurmaedje]

Regarding EcoString: I wouldn't add a dependency on it here. Doesn't seem worth it to me.

[mkorje]

Looks like there's quite a bit more going on here than in typst-layout/src/math/text.rs, e.g. the looped and chancery stuff. Would that somehow be exposed in Typst or is it for completeness?

Yes, I plan to expose these in Typst as well. At this point in time it is just for completeness though.

For the Arabic ones I plan to add to Variants the functions inital, tailed, looped, and stretched, as well as isolated as an alias for serif. Though I should note I have no idea if these should even really have their names in English, or if shortened names like tail or loop would work. I'll try to figure this all out later.

As for the chancery/roundhand stuff, we'd add scr to join cal in Typst. But there is still some work to do before it can be added (see typst/typst#5853).

[mkorje]

I've changed the to_style function to return a custom iterator type and have removed the MathStyling trait for now (see #20 (comment))

Instead, maybe to_styled could be made generic over its return type T: Default + AddAssign<&str>. Or maybe I'm just overthinking, and we can just add the dependency behind a feature flag.

Also, you can implement MathStyling for str instead, that way anything that derefs to str (such as String or EcoString) can benefit from to_styled.

Since I've removed the trait for now, this no longer applies.

Regarding the implementation, as well as having many to_{style} functions, there could also be a MathStyle::apply method to apply a style to a char.

I think in my previous reply to this I misunderstood what you were saying, sorry. I've done that now.

Also, I was wondering whether to make the to_style function a method on char instead?

I also need to double check Arabic, as for the normal style the math symbols have separate codepoints (unlike latin where the upright serif for math is the same as the standard codepoints), I think.

[MDLC01]

Maybe I did not understand the purpose of ToStyle, but shouldn't all public functions return it instead of [char; 2]?

[laurmaedje]

I agree that we shouldn't have [char; 2] in the public API. I don't think we need a public MathStyle::apply. I think we can inline its implementation into to_style.

[mkorje]

When you say not having [char; 2] in the public API, does that include the functions in mappings?

[laurmaedje]

I hadn't seen that those are public. Feels slightly duplicate to me with the enum. It's nice to have have in the type that they return just one char. I guess it depends a bit on how you plan to use them in Typst.

But, even if we keep separate functions, I think it would make more sense for the ones that can return multiple chars to return ToStyle and the other ones just a single char.

[mkorje]

Ok I've redone some stuff and more thoroughly documented/commented everything, which makes the code all much clearer I think. I also double checked the deltas and cross referenced with MathML Core. I think this ready from my end to review.

I hadn't seen that those are public. Feels slightly duplicate to me with the enum. It's nice to have have in the type that they return just one char. I guess it depends a bit on how you plan to use them in Typst.

I just decided to make those functions private in the end.

[mkorje]

Oh there is one thing left: we should maybe (?) switch alef, bet, dalet, gimel to map to their mathematical glyphs instead of the Hebrew letters. Also maybe remove shin?

Done this in the latest commit, can always undo/modify.

[MDLC01]

Suggested change

	/// The version of [Unicode](https://www.unicode.org/) that this version of the
	/// The version of [Unicode](https://www.unicode.org/versions/Unicode16.0.0/core-spec/) that this version of the

This may be a more useful link.

[mkorje]

Though that would then need updating with each version change as well. Not sure if this is really much of a point though. IIRC I basically mirrored what other Unicode crates do, including in the std library.

[MDLC01]

Well, the tuple below will have to be updated as well anyway. But alternatively you can replace Unicode16.0.0 with latest.

[mkorje]

That could also cause issues as latest might not match the version here 😅

[mkorje]

Ok I finished up the comments. Before you merge this, let me just make sure everything is working on the Typst side.

[mkorje]

I'm blocked by typst/typst#6441 now on the Typst side as #86 is causing errors... I'm too tired to try sort this out tonight, but I am like 75% confident this should still be working on the Typst side 😆

[mkorje]

I should also remove Script from the MathVariants enum as Roundhand and Chancery are already on it.

[MDLC01]

Except script is different because it doesn't specify a variation selector, right? If so, we can probably keep it (even if we don't use it in Typst).

[mkorje]

Yes, but if the font doesn't support it it just gets ignored.

I thought to remove it because we'd then not be using it in Typst, though I guess there's no harm in keeping it.
EDIT: I guess it could become a problem in the future though if we make the variant field on EquationElem no longer hidden...

[mkorje]

I added the Hebrew mappings here and then also added a check so that non-latin characters don't get styled as [c, VARIATION_SELECTOR] when requesting Roundhand or Chancery.

This is now all done, and working! (See the Typst PR)

[laurmaedje]

Thank you!