Theme accessibility

[not-my-profile]

Web accessibility is very important. For a syntax highlighting library this is particularly relevant when it's used for a website that is meant to be used by many users, some of whom may not have perfect vision and struggle to read code highlighted with themes that have a low contrast. The WCAG web accessibility standard defines the following AA success criteria:

The visual presentation of text and images of text has a contrast ratio of at least 4.5:1

Currently highlight.js already comes with several themes that meet this criteria. However most themes that come with highlight.js do not meet WCAG AA. I don't think that this is a problem per se, since highlight.js might be used just for e.g. just a personal application where the user chooses can choose their own color theme as they wish.

I do however think that highlight.js should address two things:

• If highlight.js comes with themes that do not meet WCAG AA, the Demo should raise awareness about which themes have a high contrast and which themes don't. This could be as simple as dividing the themes list in Themes with high contrast, Themes with lower contrast and linking some informative web page that explains when/why this matters.

• I find it problematic that the Default theme does not meet WCAG AA. I think either it's colors should be changed to meet WCAG AA or it should be renamed to something else (maybe Legacy Default). This is important because accessibility by default is important. (And also because the README currently suggests using default.css).

To get this started I opened the pull request #3390 which calculates the minimum WCAG contrasts for each theme and asserts that they are the same as defined in test/contrast/min_contrasts.json. (Contrasts can still be decreased but not accidentally).

[joshgoebel]

I find it problematic that the Default theme does not meet WCAG AA.

This seems like a valid criticism. I don't think we can majorly change our default theme in a minor release, but if it would only require some smaller contrast changes I could imagine slipstreaming into the next v11 release.

How far off are we exactly currently?

[not-my-profile]

The worst contrast of the default theme currently has a ratio of 2.4:1 (as you can tell by looking at the JSON file of my PR). Contrasts can be easily improved with Chrome's developer tools (which let you snap colors to the most similar color that fulfills WCAG AA). To tell how different that looks we'd have to try.

[joshgoebel]

@not-my-profile Have you visited the https://prismjs.com project yet? If you have an issue open over there (or on any other code highlighters) I'd love a link. In some quick playing around it seems 4.5:1 is actually pretty hard to hit in some cases without severely limiting color selection (I was playing around with our default theme). Before you brought it up I would not have guessed that "most themes that come with highlight.js do not meet WCAG AA"... I'd be very curious to hear some other highlighters perspectives on WCAG AA accessibility and how the standard should be applied to code highlighting (as a sub-genre of text).

Have you done any study of the full Base16 themes I wonder? They should be trivial for you to analyze. I wonder how many of those would be WCAG AA. Perhaps another project for you to look at next that could have a large impact (on many sites, editors, etc) - if you got the theme designers to care.

I think your laser focused bullet points here were good. Having a more accessible default theme certainly can't hurt - users are after all always free to discard it if they don't like it - so it forces no ones hand. I'll make an issue to tighten it up a bit for the next release, though perhaps not fully to 4.5:1... I've added a TODO for making both our light and dark default themes WCAG AA compliant to for version 12. #3219

the Demo should raise awareness about which themes have a high contrast and which themes don't

I'm thinking this could quite likely be 100% done on the client-side be analyzing the colors real-time when a theme is selected so the UI could say "high contrast" or "low contrast" or even show the range... that doesn't "divide" anything but ti does make it immediately clear at a glance what the contract of an individual theme is.

Side note: It's also impossible for us to know the size (or weight, or font) the code is ultimately going to be rendered at in the wild (our CSS doesn't force size), which obviously provides some wiggle room I think - this is obviously a "spectrum" style issue in the real world (where the themes are used) - it's impossible to look at our CSS alone and pass/fail it - you'd need to also know how it was deployed (font size, weight, font itself, etc)... I'm not sure WCAG takes all those things into consideration but obviously it's clear they all impact accessibility with regards to contrast and readability.

I say that only to point out that any contrast numbers calculate off of just the colors alone are going to be subject to some wiggle room...

and linking some informative web page that explains when/why this matters.

I'm not opposed to a small blurb in the README (or link in the demo), but I ultimately this is an implementors issue (and always has been) - doubly so if we implement the two bullets points you suggest above.

---

So if your goal here is having some immediate impact:

• I've just created Improve accessibility (slightly) of default light and dark themes #3393 for improving our default themes, which should land in the next release or two, even if it's not "perfect" WCAG AA. This will improve accessibility for anyone who version bumps and uses our default themes.
• How about providing a suggested blurb for the README about theme accessibility... (or a PR with just that single doc fix)
• I'll still circle back to your other PR, but right now I'd be a lot more open to doing this high/low contrast analysis on the client-side in the Demo live (as suggested above). IE, simply telling people when clicking thru random themes how accessible the theme they are looking at may or may not be.

[not-my-profile]

If you have an issue open over there (or on any other code highlighters) I'd love a link.

I have previously raised the same issue with Pygments (pygments/pygments#1718) and they have been very responsive.

In some quick playing around it seems 4.5:1 is actually pretty hard to hit in some cases without severely limiting color selection (I was playing around with our default theme).

As I mentioned with the Chrome dev tools getting WCAG compliance is trivial, you just inspect the offending text and click a button. If that results in two different style colors looking too similar you'll have to change them, but it's really not that hard.

Have you done any study of the full Base16 themes I wonder?

No, I haven't.

I've added a TODO for making both our light and dark default themes WCAG AA compliant to for version 12.

Thanks!

I'm thinking this could quite likely be 100% done on the client-side be analyzing the colors real-time when a theme is selected so the UI could say "high contrast" or "low contrast" or even show the range... that doesn't "divide" anything but ti does make it immediately clear at a glance what the contract of an individual theme is.

You have 242 styles. I really don't think that users should have to click through each of them to find the ones that have a high contrast. I believe highlight.js should make it as easy as possible for implementors to pick an accessible style (if they choose to care about that). (Hence my PR).

It's also impossible for us to know the size (or weight, or font) the code is ultimately going to be rendered at in the wild (our CSS doesn't force size)

Yes for 18 point large text (that's 24px) a contrast ratio of 3:1 suffices to meet WCAG AA. In practice that doesn't apply to most highlighted code because:

How about providing a suggested blurb for the README about theme accessibility

Sure that wouldn't hurt. I still think that showing that information on the Demo is more important since any note in the README can be easily missed.

[joshgoebel]

I wrote a small Python script to check how Pygments styles fare when judged according to the WCAG contrast standard.

Was this not possible with our CSS because of it's complexity? Just posting a simple list like you posted on the Pygments thread would have been super useful (to kickstart the discussion) and sounds like it might have required far less of your time than your original PR.

you just inspect the offending text and click a button

Yes, the key part was "without severely limiting color selection"... I was playing with another tool and it was easy to move the slider but very quickly all the colors turn VERY dark... it seems a very limited dynamic range that fits in 4.5:1... (when your BG is white/light grey)

[Re: Base16 themes] No, I haven't.

Perhaps you'd point your script at them? Here are ones built with our template:

https://github.com/highlightjs/base16-highlightjs/tree/main/themes

These include only #000000 style html colors I believe, so I think you should find them easy to parse... Would be curious to see the results if it's easy for you to get them.

You have 242 styles. I really don't think that users should have to click through each of them to find the ones that have a high contrast. I believe highlight.js should make it as easy as possible for implementors

I take your point. I think we're coming at this from very different perspectives. I imagine most implementors would find a theme they liked first visually - which kind of requires "clicking thru them all"... then perhaps they check how accessible it is... perhaps they'd be happy with 4.3:1... ...where-as you imagine someone saying "I deeply care about accessibility, show me only high contrast". So from my perspective it sounds a little less ridiculous. :-)

If we're going to surface this in Demo I think we we should calculate this during the Demo build process (with a smart analysis tool that doesn't force CSS rules on us and actually examines the final built CSS, not the raw source), not during CI.

Yes for 18 point large text (that's 24px) a contrast ratio of 3:1 suffices to meet WCAG AA. In practice that doesn't apply to most highlighted code because:

I was just pointing out it's technically a "sliding scale" as the WCAG even seems to acknowledge: https://www.w3.org/WAI/WCAG21/Understanding/contrast-minimum.html#dfn-large-scale The sudden jump from 3 to 4.5 is a bit well, sudden... it'd be more realistic to scale the contrast ratio with the text size... The point of a standard is having a standard, so given an exact example you can indeed say pass/fail... but I'd suggest that if we're being honest then 23px large text that's 3.2:1 is probably within the "spirit" of the standard, even if not the letter. :-)

[joshgoebel]

Ok here is what I whipped up really fast on top of our existing theme checker: #3394

It's not perfect (ie, dealing with all the CSS variations) yet, but even so it's super useful stats... and this feels much more in line with the type of tooling we'd prefer to deploy. It'd be pretty easy to break this out into a tiny analysis lib that just takes the CSS as input... and then provides a stats object with best/worst/median/etc... which could be directly plugged into the Demo builder.

So... given that this seems easy enough to iterate on... the next step would be decided how Demo should leverage this information.

[joshgoebel]

@Hirse You've played with the Demo stuff a bit I think... any thoughts on UI for this? We need a segmented list or toggle or filter or something... Is it just a dichotomy we want to offer- high vs low contrast? Or perhaps we just leave it as is but add a tiny WCAG badge next to "winning" themes... though I imagine the original requestor here would prefer to see them grouped?

@not-my-profile Did you have any specific ideas of how to surface this on Demo? Should I assume you think showing the actual WCAG numbers is important? If so, which? Worst? Average? Median?

I'm still also struggling with which exact numbers to use for "grouping"... to me a theme isn't bad just because a single rarely-used scope is a bit low contrast if 99% of the text is still high contrast. That's something I think we should make an effort to take into account when segmenting themes. If someone wants to be a stickler about it - that's a good reason to surface the actual WCAG numbers somewhere.

I think we should weight plain text and comments higher than other scopes. Plain text is going to be the majority and faded comments are likely to be the worst abuse in many themes.

[Hirse]

Thanks @not-my-profile for starting this conversation. I have been thinking of theme accessibility before, but never got started.

@joshgoebel, I was thinking of exposing accessibility classification (and light/dark status) as icons in the theme list. Here's a quick example using emoji where the wheelchair stands for AA/contrast>=4.5 and the guide dog for AAA/contrast>=7.

The actual contrast value is available as a tooltip.

We could then add a toggle to only show accessible themes.

(I would like to find better symbols, preferably something like , but unicode is just so easy for testing.)

For classification, I used the values calculated in #3390. Generally, I think we should be strict and use the min-value when using the calculation in #3394.

[not-my-profile]

I agree with @Hirse that you should be strict.

to me a theme isn't bad just because a single rarely-used scope is a bit low contrast if 99% of the text is still high contrast. [..] If someone wants to be a stickler about it [..]

There are certain service sectors where websites are legally required to meet WCAG AA. It's also not uncommon for WCAG AA compliance to be part of a contract for commissioned work. 99% does not cut it in those cases.

I don't think you should display a 🚫 for each stye that doesn't meet AA. I think it makes more sense to display a green AA badge or a green AAA badge to the right of theme names for themes that meet that criteria.

[joshgoebel]

I don't think you should display a 🚫 for each stye that doesn't meet AA.

Agreed, this isn't a public shaming.

There are certain service sectors where websites are legally required to meet WCAG AA.

I'm not suggesting we should misrepresent whether a theme is WCAG AA or not, but we could have our own designation too... so themes could be high contrast (in our humble opinion), AA, or AAA... but no use in arguing about that until we see what the actual breakdown looks like in the first place. :)

---

Yeah, maybe instead of icons just a little <span>AA</span> style pills could work... I think that would be an easy start.

[joshgoebel]

I'm not sure I have time to push this to the home plate fully right now but I'll try to finish up the "rough shape" of things and then update my PR and someone else could pick it up and run with it the rest of the way or it could wait until I have time to circle back.

@Hirse Maybe you'd be interested, all you'd need to do is work on the client-side display/filtering pieces... should be pretty simple. I think we could have the same "pills" (vs icons) beside the title and those serve as button "selectors" to filter for just themes matching that criterion ...

Now I'm thinking we could have "Dark" and "Light" filters also since that out to be easy to do while we're analyzing the colors, but we'll see. :-)

[joshgoebel]

A beginning. Themes in the Demo now have badges. I updated my PR.

[joshgoebel]

@Hirse The whole A display:block will have to be revisited though. :-) Let me know if you might be interested in doing the front-end UI work on this (filtering, etc).

If we're going to include a small blurb or link to a page on accessibility we should decide where to add that... I think it's fair to have a tiny call out that explains what the AA and AAA badges mean and that's where we'd include this.