JAWS not reading MathML present in HTML form element labels

[dlhubble]

Summary

Some HTML form elements may be frequently used in conjunction with math. One example is multiple-choice math questions with answer options that are mathematical and presented as radio buttons. In such cases, the mathematical answer options are naturally placed in the button label. However, JAWS does not read MathML within labels, which leaves the user with only the button itself being announced. This problem also exists when MathJax is used, but since MathJax defaults to passing on MathML to JAWS, it is expected that issue will resolve when solving this MathML issue.

1. Go to https://indiana.sharepoint.com/:u:/s/O365-UITSATAC/EZt6aBlfITFJjon__TEq6DkBJoz58Y4HRSuxq0dLogSyug?e=Jg1r9O
2. Navigate to the radio button section.
3. Listen for the MathML that is present in the radio button labels to be announced.

Expected result

When JAWS announces a radio button with math in the label, one would expect to hear: “Option 1 square root of 1 i math content radio button not checked 1 of 2”

Actual result

When reading a radio button with math in the label, JAWS instead announces: “Option 1 radio button not checked 1 of 2” completely omitting the MathML.

Example

https://indiana.sharepoint.com/:u:/s/O365-UITSATAC/EZt6aBlfITFJjon__TEq6DkBJoz58Y4HRSuxq0dLogSyug?e=Jg1r9O

Additional Information

JAWS version and build number

JAWS Version 2021.2103.174 ILM

Operating System and version

Windows 10 Enterprise x64 - ConfigMgr - v1.0 - 5.21.2018

Browser and version:

• Microsoft Edge version 89.0.774.75 (Official build) (64-bit)
• Google Chrome version 89.0.4389.114 (Official Build) (64-bit)
• Firefox 87.0 (64-bit)

[JAWS-test]

MathML is no longer supported by Chrome and Edge. I.e. in the current versions of the browser the problem does not occur anymore - the label text is output completely, but not correctly (only as text, not as mathematical formula).

In Firefox the problem occurs, but it is not a JAWS bug, but a Firefox bug, because the content of the math element is not transmitted to the Accessibility API as part of the label. Therefore, the problem also occurs with other screen readers such as NVDA.

To be quite precise: it is actually an Accname specification problem, because the math element only has a label if it is explicitly labeled. Text content does not count as a label. See w3c/accname#120

[JAWS-test]

The problem occurs with all form field types. However, it is particularly problematic with radio buttons and checkboxes, because only the label is output with these form element. With all other form field types (e.g. input fields), the label is output separately when reading linearly with the arrow keys and is thus fully perceptible at least with the virtual cursor (even if only as text and not as a label).

[brichwin]

I thought JAWS supported reading MathML. Regardless of if Chrome or Edge supports MathML natively, MathML rendering layers like MathJax and KaTeX expose the MathML to AT to be read.

For example, the default behavior of MathJax v3 is to let JAWS parse and read the MathML - so this would be in any web browser, not just Firefox. JAWS reads and explores math in Chrome just fine in many cases .... so JAWS does support reading MathML in Chrome.

[JAWS-test]

My test found: JAWS does not support MathML, but only the code generated by MathJax. This then works in Firefox and Chrome/Edge.

When I test the above example with MathJax, I find that in Chrome/Edge the mathematical content is output as plain text and not as a mathematical formula, although the formula itself is output correctly (if it is not part of the label). The difference with Firefox would be that there the MathJax content is not output at all if it is part of the label

[brichwin]

When you view the HTML page the submitter provided in Chrome, doesn't the math under the first level 2 heading read correctly? That page does not use MathJax or any other math rendering JavaScript layer. So.... JAWS does read MathML from web pages, right?

[brichwin]

Peter Krautzberger (one of the MathJax creators) confirmed to me that the default behavior of MathJax is to:

The default settings actually add visually hidden MathML into the page for AT (not the speech text that MathJax can provide). So the bug is likely connected to JAWS MathML support.

[JAWS-test]

@brichwin

I have tested it again. Unfortunately, I got different test results that are not consistent. In my last test JAWS with Chrome also output Math correctly as formula without MathJax. In the test before that, it made a difference if the xmlns attribute was on the math element or not - very strange.

Anyway. The current test results are:

• JAWS can correctly output the math element as a formula if it is text. This is true for Firefox and Chrome, although in Chrome the rendering is not correct.
• JAWS cannot output the math element correctly if it is not text but, for example, a label. In Firefox the label is not output at all, in Chrome it is output as text and not as a formula. Whether this is a JAWS bug is debatable. The problem is that labels are always output as unstructured text because that is how the Accname specification says it should be and that is how the browsers' accessibility API handles it. That is, JAWS receives incorrect information from the browser and outputs it accordingly. This can be seen well with F12.

Imagine you have an input field whose label consists of a heading, a graphic, and a list. Even in this case, a screen reader will only ever output the text content of the heading, graphic (alternative text), and list, but never the structure and role of the elements. The same is true for math elements.

[brichwin]

In many cases, the JAWS software augments the accessibility API with information from the DOM. For example, an empty link that lacks any other potential label (ARIA, title attribute, etc.) will be announced by some portion of the URL found in the link's href attribute.

I've assumed this non-standards behavior is a feature of JAWS where it is attempting to at least provide some identifiable moniker as a last resort to help the user.

For example, JAWS still provides a "name" of "long/urlAndFileName" for the link in this code :
<p>Here is an empty link: <a href="https://iub.edu/really/long/urlAndFileName.html"></a>
When using the Accessibility inspector shows only: Role: link, Focusable: true, Name: ""

If FreedomScientific wanted to provide a consistent level of support for reading mathematical content, it seems a similar model of reaching past the accessibility API could be employed to scan the DOM for the presence of math and augment the accessible name with speech text representation of the MathML content. This might be the only way such an issue would get resolved as this issue

Is there a place to make a JAWS feature request?

I

[JAWS-test]

@brichwin You are right: JAWS can access the DOM if it wants to. So unlabeled form fields are also output with the text before or after them. But I think the problem here is that JAWS only does this when a link or form field is unlabeled, i.e. as a repair mechanism for incorrect source code. But if JAWS gets a correct name via the Accessbility API, it doesn't bother to parse the DOM. And here that is the case: both Firefox and Chrome submit a name (albeit a different one) via the API, so JAWS trusts that it is correct.

[JAWS-test]

I think that ticket should just stay open, then Freedom Scientific can check if they want to improve something in their product.

[stevefaulkner]

@brichwin @JAWS-test I had a look at this and think the issue is that the label content is flattened to a text string sans math output. I believe this an artifact of what is exposed in the acc layer.

Anyway, a work around I found:

1. move the <math> element to left of the input as a sibling of the label
2. add tabindex="0" to the <math> element.
3. CSS to place the math content as desired
   a codepen - example

When the user arrows into the fieldset the math content is announced. arrow again to a radio button the label content is announced.
This only works in Virtual cursor mode, math content is not announced in PC cursor mode.

[brichwin]

@stevefaulkner Thanks for taking a look at it!

However, I don't think it's a valid solution as it runs against expected behavior. The expected behavior being that traditionally when content is related to a radio button or checkbox, it appears both visually and in the DOM immediately after the radio button or checkbox - not before it.

When a non-visual user reads though the solution, they could very easily assume the math goes with the prior radio button (square root of 2 i math content goes with Option 1). The non-visual user w/o any usable vision will not have benefit of the CSS produced visual appearance. The expectation that moderately skilled screen-reader users would successfully take on the extended cognitive burden of having to figure out this non-traditional approach in the context of taking a high stakes exam seems unreasonable to me.

Since the math content in the offered solution reads out immediately after the prior button's label and before the radio button it is meant to be paired with, a non-visual student could easily say they were mislead into selecting the wrong answer and both the screen-reading experience and the code would justify their claim.

[stevefaulkner]

@brichwin yeah, understood, unfortunately there is not a better solution available currently, I don't think it practical currently to include MathML as part of a label, as the accessible name calculation is based on plain text, not rich text.

[brichwin]

The only solution we could come up with is this:

When needing answer options with MathML, provide all of the answer options in the question area as a list. with labels that correspond to the labels in the answer selection radio buttons or checkboxes. This allows for the math in the answer options to be explorable and read out correctly.

For example (pretend MathJax is rendering the LaTeX as MathML):

<form action="/action_page.php">
  <h2>Question 1</h2>
  <p>What is \(\sqrt{1i+2i}\)?</p>
  <ol type="A">
    <li>&nbsp; \(\sqrt{i}\) </li>
    <li>&nbsp;  \(\sqrt{3i}\) </li>
    <li>&nbsp;  \(\sqrt{1i}+2i\) </li>
    <li>&nbsp;  \(\sqrt{3+i}\) </li>
  </ol>
  <fieldset><legend>Choose Answer for Q1: </legend>
    <div><input type="radio" id="q1a" name="q1" value="A"><label for="q1a">A</label></div>
    <div><input type="radio" id="q1b" name="q1" value="B"><label for="q1b">B</label></div>
    <div><input type="radio" id="q1c" name="q1" value="C"><label for="q1c">C</label></div>
    <div><input type="radio" id="q1d" name="q1" value="D"><label for="q1d">D</label></div>
  </fieldset>
</form>

See it on codepen: https://codepen.io/brichwin/pen/VwpdyjY
The   in each list item is for another JAWS bug where JAWS doesn't reliably read math content when it appears first in a list item.
Note: Unless punctuation verbosity is set correctly, JAWS will omit announcing the plus operator in the above example.