Ligatures break metric compatibility #31
Comments
|
ff, ffi, fi, fl, ffl. These should strongly not ligate by default, it’s obviously wrong for a monospaced font. |
The whole point of this font set is to serve as a replacement of the PS L2 core 35 fonts, and Courier is one (well, four) of them. @chris-liddell Any opinion (from you or Artifex) on this? Could we at least get to choose which variant (with or without lig) to use? (I mean like variant of font file, not relying on fontfeature whatsoever in fontconfig.) |
|
Sorry for taking so long to respond. Basically, we've tried to refer to URW++ to this (and the other issues on here), and I was planning to reply here when we had a response from URW++. But they haven't gotten back us, and honestly, I forgot. I assume this is relates to the OpenType/Truetype version of the font. |
|
Well only the OTFs. I'm not even sure if TTF support ligature anyway. I mean, at least it doesn't even seem like you can have it explicitly enabled with the CSS FYR the current |
|
Hmm actually according to the tech specs on MyFonts, Helvetica from Linotype does support Also, both Nimbus Sans and Nimbus Sans L on MyFonts appears to support On the other hand, there's a Courier PS from Monotype that supports It seems that the best alternative available is GNU FreeFont. FreeSans supports |
|
And the
|
|
Well, there is an argument that they are described as metric compatible, not fully compatible fonts. By the looks of it, disabling ligature substitutions would produce metric compatible results. But, frankly, we're not the source of these fonts, and we can't do much more than pass on the problems to URW++. We don't have that kind of font expertise in-house. |
|
Isn't there any fonttools or gftools command that removed ligature definitions from a font file? |
It's true, TTF and T1 fonts don't feature ligatures. TTF also doesn't support cubic bezier splines, so they render slightly worse (or require much larger glyph descriptions to get an indistinguishable quality). Type 1 and OTF do support cubic splines and use them here. |
https://adobe-type-tools.github.io/afdko/ That said, it's still tricky. AFDKO barely allows such edits. FontForge deliberately recompiles the entire font from scratch and applies a number of changes to align with its taste of good quality. To list ligature sets, use To edit a font, convert it to XML, edit it, then convert it back to OTF: Ligatures aren't hard to find and remove, just search for "liga" or "14" (including quotes). However, there are two catches:
That said, I'm also researching yet another strategy. |
Hm, as a last resort we could still fall back to packaging the TTF variant if its feature set aligns more closely to the original T1 fonts. |
|
TTF and T1 formats don't have such feature sets, querying these variants with The OTF format has more features than just ligatures. There's also a mechanism to replace |
AFAIUI, in this specific case the OTF fonts aren't supposed to have any more features than the T1 fonts they are meant to replace. Anything else will be another bug in the OTF fonts. |
|
Alright, Nimbus Mono fonts should be fixed in the attached commit. Instructions on how it was done in the commit message. I verified the result with all tests I could imagine, please test yourself with what you can imagine :-) 0001-Remove-ligatures-from-NimbusMonoPS.patch.gz Please apply as following, this would save me all this pull request mess. For verification after applying the patch: |
If one wants T1 fonts, it's perfectly fine to use these T1 fonts. T1 even uses the higher quality cubic bezier outlines. Nimbus Mono improved a lot over the years. The version committed first in this repo had just 233 glyphs, now there are 855 glyphs. Some of these improvements took advantage of technologies which older formats like T1 and TTF don't support. The issue opener asked for metric compatibility with Courier. That's what it is now. |
|
@merchantsedition Instead of providing a binary patch, could you probably recite the procedure used to create the patches fonts here instead? |
|
It's a text patch, just compressed for convenience. This is what it writes about the procedure used: |
|
BTW., if you really want to remove all substitutions, it's as simple as (untested): P.S.: I won't do this, because I prefer to change only things where I can prove misbehavior and also test the result. |
|
@chris-liddell @deekej |
|
Hello @fabiangreffrath - unfortunately I no longer work as a package maintainer in Red Hat, and in my spare time I don't have capacity to incorporate this into Fedora either. I suggest contacting the current maintainer for urw-base35-fonts in Fedora if you wish to see these changes there... ;) |
It's not that I want to see the patch applied in distribution packages, else I would have already applied it to the Debian package. I want it to be applied upstream and then migrate to distributions the natural way. |
|
Sorry for the lengthy lack of response, we've been trying to get some kind of communication "upstream" and have basically had no luck. So, when I have some time available, I will look at incorporating the patch @merchantsedition supplied, and some other changes. I have, frankly, been extremely reluctant to do this, because the previous time the URW++ fonts were forked, it caused us no end of headaches, but I think we're out of options :-( |
I don't mean to jump in late to a discussion here, but I'm noticing several indicators that there's some confusion of terms in the thread, which has the potential to cause unwanted ill effects for eventual users downstream.
Whether a particular So the blanket notion that "monospaced fonts shouldn't have ligatures" is not true. The slightly more focused "monospaced fonts should not have GSUB substitutions active by default that change the width of the sequence" is maybe a little more defensible, but then again, reading the thread it sounds like that is purely a matter of taste, I suspect coming from folks who consider monospace fonts as being exclusively used for terminals or code editing. That's not the only use case. That's what the In both cases, it's extremely dangerous to chop out GSUB features and it can break people's documents. Requesting the change be made to an "off my default" GSUB lookup by the upstream font publisher is half of the solution. The other half is that code-listing widgets and terminal apps should have optional GSUB features like There's some decent technical rabbit-hole reading to be found at https://simoncozens.github.io/fonts-and-layout/features.html that people might find helpful. Also, I know I've already gone super long and doubt anyone's still reading, but GSUB and GPOS are part of Open Type Layout, which is an extension, and both GSUB and GPOS are 100% valid tables in a file with a .ttf extension. OpenType as a whole is a superset of TrueType (meaning every TrueType font is also an OpenType font; they're not mutually exclusive), and the filename extension doesn't tell you which tables might be inside, especially with OpenType Layout and the other extensions to the original published spec. |
Not so sure about this. I mean, I can't rule out that a fraction of people in the world might be into appearance like " a ffi n i t y", but I'm also quite sure most sane people don't have such a "taste" (more like a fetish, if I may). Or maybe what you mean is that the ligated ffi should have the width of three characters instead of one? Otherwise it's a renderer's fault? Anyway, this really isn't a "background" or "context" thing. It's just the same when it comes to website or PDF. (Actually, I wouldn't have even participated in this discussion if it's a terminal or code editor thing. Does anyone even use nimbus mono for that anyway?) And the examples you have given (Arabic, italic latin) don't seem really persuasive, as you just seem to assume that what's appropriate for them should apply in monospace font as well. (I mean, is monospace even a thing when it comes to Arabic?)
In my opinion, the fact that it's "not knowable" (or, "can't be guaranteed") is itself quite a strong reason that monospace font should not support that at all. Otherwise, what's the point / definition of "monospace"? (Which is why I feel like, yeah maybe there's a way for a program to find out whether a font is of the monospace family and determine whether ligature is enabled based on that, but it doesn't sound like its responsibility either.) (I can't help but think of the "origin" of monospace font. Isn't that typewriter? And is ligature a thing in that? If so, perhaps it tells us how ligatures should look like / be done in monospace font?)
I'd say only the other way round is true. But meh. It's not really relevant to the discussion, is it? P.S. I am not replying to like make sure this "downstream patching" gets in at the end. To be frank I don't really like the idea, as it does smells like "something not gonna end well". Neither am I trying to stop it from happening / intervene though. |
They still can use the ligature if they want to, because application settings and markup permit them to activate it. Breaking the font for them is what takes that flexibility away.
This is pejorative straw-man stuff and not germane to a serious discussion.
No, it's 100% a context thing. Browser or DTP app included. Did the original user see the "fi" ligature they don't want inside a <tt> or <code> element in Firefox? Then it's a Firefox bug. Did they see it in an inline-code-block in their Markdown editor? Then it's a Markdown editor bug. Did they see it in vim? Then it's a vim bug. If you don't know where they encountered it, then you don't know where the bug lies. That's why breaking the font for all users isn't the solution. The purpose of GSUB's suite of lookup tags is that applications can set the combination of feature settings appropriately for individual elements. E.g., tabular numerals ON by defaults in spreadsheets, but OFF by default in text-paragraphs.
I am talking exclusively about monospaced fonts.
Yes. If you are genuinely needing to ask that, then that suggests you are not yet sufficiently brushed up on the background of how and where OpenType features are used in fonts and why. You really would do well to read the specification. Even the intro to OpenType Layout would probably make a lot of things click that are jumbled elsewhere in bug reports: https://learn.microsoft.com/en-us/typography/opentype/spec/ttochap1 Just as monospaced italics are a thing. Monospace italics and monospace bolds & other weights are how code-highlighter engines distinguish different semantics. Additionally, ligature-substitution lookups are how all those "coding ligature" fonts popularized in the past few years function. I don't find many of those substitutions useful, and some of them (like double-wide greater-than-or-equal) I find downright unappealing. But they can be optionally activated or deactivated because of the GSUB feature-tag mechanism. Which again is why trying to hack them out of the font binary in the upstream package is not the solution; user X might not like the ligatures, but breaking the font for all users causes harm to them.
Well, it's not a matter of opinion, it's a specification, and you have it 180 degrees backwards in that statement. More importantly, here:
It is relevant. I brought it up because of this in comment 1265993694:
which is incorrect. Even outside of GSUB and GPOS (which .ttf files can — and do — include), .ttf fonts can include ligatures in other tables. Most often the
On that we fully agree. :) |
On a different note, FontForge can't really be relied upon to do surgery like this on a binary font file. GSUB and GPOS lookups can chain and "reverse chain" and be defined with context referring to other lookups, and FontForge doesn't handle that. Removal could break those interdependencies and FontForge doesn't attempt to do constraint-satisfaction stuff like would be required to close all the loops. I wouldn't particularly rely on AFDKO for that either, but mostly because it's less used and might not have been tested. |
I think it was a <pre> element, but it's been about a year and a half since I reported this so I could well be misremembering. But you're mischaracterising my report. I don't care whether I see a ligature, I care that the layout gets all messed up as a result. A font that is claimed to be metrically compatible with Courier needs to have the same metrics as Courier regardless of whether the applications enable or disable ligatures, because the font substitution will be applied regardless of whether ligatures are enabled, and right now, when ligatures are enabled, they aren't, Nimbus Mono-with-ligatures has different metrics than Courier-with-ligatures. If Nimbus Mono were to have an fi ligature that has the same width as an f character followed by an i character, that would be just fine. |
I get that completely. I'm saying that the problem you're describing is one of behavior, not of metrics. The metrics are defined as widths & advances of each glyph, in the
If Courier also had an fi ligated character that had that same width, then the two fonts would still be metrically compatible. The difference in the behavior is about whether or not this substitution happens when you use it, not about the widths and advances. If Nimbus Mono and Courier have fi ligated characters that are different widths, then the two fonts have different metrics, regardless of whether or not each one performs the "f, i"->"fi" substitution always/sometimes/never. The test case there would be if somebody had a document where they intentionally put in the "fi" ligature by its Unicode codepoint (U+FB01): if the two fonts have different widths for that, they're not metrically compatible, and that person would see it because their document (or screen or whatnot) would have its text reflow. That's the difference between metric compatibility and behavior issues. |
What I really mean by background / context does not matter is, I think when it comes to monospace, it's not about whether "some of us are used to how it should look like", or whether a font format spec states that something is supported or not, or whether the font is used for certain "semantic" reason. Rather it's a matter of the definition of monospace. It's quite a intuitive / simple math thing as far as I can see -- regardless of whether ligature should be done / is desired, the first rule is that the after ligation "it" should be still the same width as others, with "it" With that said I'm not ruling out it's not something to do with the font, but perhaps yes, everyone just screwed up historically in monospace rendering. But at the same time, IMHO whether we can / should just "cut off" ligature in Nimbus Mono can still depend on characters it has / would ever cover, even if the "dangerous / wrong" case you mentioned are real.
And this is probably the most important point I would like to emphasize as well. Personally speaking I don't even care / know much about metric compatibility. The real problem is that if ligated ffi is not of the same width as not-ligated ffi, then something (renderer or the font) must be wrong. (Although to be frank, I'd probably find it odd even if the width is the same, but that I can sort of accept it to be "a matter of taste". For now. I'm still thinking about the typewriter / origin matter.)
I think that's why "the other way round" thing matters. As you said, OpenType is superset of TrueType, which means there's still a set for TrueType. And regardless of whether that has something to do with the implementation of font renderer or so, or whether the ttf available here really don't have the ligature table whatsoever, de facto speaking, the issue is apparently not observed in the ttf here. I think what the comment really means is, "if we can survive with the practically no-ligature ttf files here, we would be able to survive with such otf as well. (I don't think its point is about "right or wrong". Btw, AFAICT it's undeniable that the ttf here are not just the otf with different extension name.) |
I'm not too interested in a semantics game, but fine, let's go. A good starting point is always Wikipedia, and following its sources. It says:
Its sources support this definition:
It is not just about the metrics of individual glyphs. If different sets of ligatures causes text to reflow when switching between fonts, the fonts are, by this definition, not metrically compatible. And that's exactly what's currently happening with Courier and Nimbus Mono, because Nimbus Mono uses ligatures where Courier does not, and Nimbus Mono's ligatures are not of the same size as Courier's non-ligature characters. This is the definition of metric compatibility I am using. If you object to this definition and want to use a different term that means the same thing, I have no issue with that, but I have not seen an alternative single term that covers all this and only this in your message, and given that others are using metric compatibility exactly the same way as I am, I see no issue with continuing to use it the way I am. |
For a bit more context, someone had created a preformatted plain text table using Courier, like so: This rendered badly when Nimbus Mono was installed, because |
|
Ths is not about personal taste or "getting used to ligatures". Your app enables ligatures -> you render text with Courier -> you won't see ligatures It's as simple as that. |
|
To throw another spanner into the works here, the claim about "metric compatibility" really only applies to the Type 1 fonts and their compatibility with the Postscript Level 2 font set, particularly being used in a Postscript/PDF context. And since the way both Postscript and PDF can sort of use OpenType fonts means GSUB and other "advanced" features don't apply, that compatibility remains in that context. @n8willis does raise a valid point, too, that these fonts have been out there and being used in their current state. Chopping them around could well cause inconvenience to users who have documents that rely on their current form. |
|
I shall be blunt since there’s so much prevarication and attempted justification going on: the current behaviour is blindingly obviously a bug and undesirable in absolutely every case, no exceptions, and no one in their right mind will be depending on the current form or do anything other than breathe a sigh of relief if it’s fixed—quite apart from it being difficult to truly depend on the current behaviour. (But I should clarify that I’m speaking of Nimbus Mono here. I don’t venture anything with respect to other fonts, and they weren’t the subject of the initial report; if there’s any discussion to be had on any other fonts, separate Nimbus Mono from the rest.) |
|
The archlinux wiki describes a workaround for this bug. Also, I agree with @chris-morgan, and will add that on top of looking like total dogshit, it misaligns any code or ASCII art that assumes "ffi" occupies three spaces. Fix it! |
|
The workaround does not work for me. Thus, I created a PKGBUILD that packages the TTF variant and let it replace the original package. And I recommend all distributions do that until the OTF version is fixed. |
|
A fix was already provided in the comments by @merchantsedition in #31 (comment). It still hasn't been applied, but for people looking to fix this locally on their systems, I would suggest just going with that. |
Nimbus Mono is supposedly metrically compatible with Courier, but Courier displays 'ff' as two separate 'f' characters, Nimbus Mono prints it as a ligature that takes up the same space as a single 'f' character. Likewise for other ligatures. When Nimbus Mono is used as a substitute for Courier on the assumption that they are metric-compatible, it breaks layouts.
Should the ligatures be disabled (except when they are specifically requested, e.g. when U+FB00 ('ff') is used), or should the font instead not be treated as metric-compatible? If you prefer to leave the ligatures as they are, please let me know and I will instead file bugs against those places that list the font as metric-compatible.
The text was updated successfully, but these errors were encountered: