Skip to content

[Sync to prerelease] Add an axe baseline for known third-party and upstream findings - #2260

Merged
cwickham merged 1 commit into
prereleasefrom
sync-2208-to-prerelease
Oct 2, 2026
Merged

cwickham merged 1 commit into
prereleasefrom
sync-2208-to-prerelease

Conversation

@github-actions

@github-actions github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor

Description

Sync of #2208 to prerelease.

Original PR

Add an axe baseline for known third-party and upstream findings

Summary

This PR adds _axe-baseline.json, the baseline file for quarto call axe (quarto-dev/quarto-cli#14815). A scan compares its findings with the baseline. It reports only the findings that are not in the baseline as new.

The baseline has 128 signatures. Each one is a finding that quarto-web cannot fix in its own content, a false positive, or an intentional example:

  • third-party: the markup comes from a library, for example Observable Plot or leaflet.
  • upstream: the markup comes from Quarto, and a quarto-cli issue tracks the fix.
  • false-positive: axe reports a defect that a user does not see.
  • intentional: a docs example shows the defect on purpose.

Each entry has a note that gives the cause, the owner, and the condition to remove the entry. Each note is complete by itself, so the context stays if another entry is removed.

Most entries apply to all pages. An entry is limited to specific pages when its signature can also match a real defect in quarto-web content. Then a new defect of that type on a different page still shows as new.

Entries

Upstream: Quarto

Signature Count Issue WCAG 2.2
scrollable-region-focusable :: #cb (all pages) 1 signature quarto-dev/quarto-cli#14378: scrollable code blocks have no tabindex. Fixed by quarto-dev/quarto-cli#14816 2.1.1 Keyboard
Other scrollable-region-focusable code blocks and cell outputs (only the pages that have them) 36 signatures, 64 instances quarto-dev/quarto-cli#14378. Fixed by quarto-dev/quarto-cli#14816 2.1.1 Keyboard
aria-allowed-attr on collapsed callout headers (all pages; .collapsed only on the four docs/get-started/authoring/ pages) 3 signatures, 13 instances quarto-dev/quarto-cli#4934: collapsed callouts use a <div aria-expanded>, not the disclosure pattern 4.1.2 Name, Role, Value
image-alt on {{< placeholder >}} images (docs/authoring/placeholder.html only) 2 signatures, 2 instances quarto-dev/quarto-cli#14769: the shortcode makes an <img> with no alt 1.1.1 Non-text Content
aria-required-parent :: #ref-xie (docs/authoring/article-layout.html only) 1 instance quarto-dev/quarto-cli#14755: margin citations keep role="listitem", but Quarto removes their role="list" parent 1.3.1 Info and Relationships
frame-title on code-preview iframes (docs/presentations/revealjs/index.html, advanced.html, and docs/authoring/code-annotation.html only) 18 signatures, 35 instances quarto-dev/quarto-cli#14770: code-preview makes an <iframe> with no title. Fixed by quarto-dev/quarto-cli#14933 4.1.2 Name, Role, Value
meta-viewport :: meta (all pages) 1 signature quarto-dev/quarto-cli#14952: the revealjs template sets maximum-scale=1.0, user-scalable=no on every deck 1.4.4 Resize Text
color-contrast on faded lines in revealjs line highlighting (docs/presentations/revealjs/examples/line-highlighting-1.html and line-highlighting-2.html only) 3 signatures, 28 instances quarto-dev/quarto-cli#14959: code-line-numbers fades the other lines to opacity: 0.4. No opacity value meets 4.5:1 and still shows the highlight 1.4.3 Contrast (Minimum)
aria-prohibited-attr on revealjs code line numbers (docs/presentations/revealjs/demo/index.html, demo/mini/auto-animate-code.html, examples/code-echo.html, and examples/line-highlighting-1.html to line-highlighting-4.html only) 2 signatures, 87 instances quarto-dev/quarto-cli#14961: revealjs removes the href from line-number links, so their aria-label is on an <a> that is not a link 4.1.2 Name, Role, Value
scrollable-region-focusable on revealjs slides, the slide menu, and a code block (body, #slide, #scrollable-slide, .active-menu-panel, .latex; 26 revealjs decks only) 5 signatures, 30 instances quarto-dev/quarto-cli#14817: revealjs slides get overflow-y: auto, and the arrow keys move between slides instead of scrolling, so keyboard users cannot reach overflowing content 2.1.1 Keyboard
color-contrast :: #2a76dd on #ffffff (docs/presentations/revealjs/demo/index.html only) 1 instance quarto-dev/quarto-cli#14962: the default revealjs $link-color is 4.44:1 on white, which fails for small text such as the footer 1.4.3 Contrast (Minimum)
color-contrast :: #2a76dd on #bbbbbb (docs/presentations/revealjs/examples/tabset.html only) 1 instance quarto-dev/quarto-cli#14963: below 30em, the selected revealjs tab has a #bbbbbb fill, so its text is 2.31:1 1.4.3 Contrast (Minimum)
link-name :: .slide-menu-button > a (all pages) 1 signature quarto-dev/quarto-cli#14249: the slide-menu button is an <a href="#"> with only an icon, so it has no accessible name. Fixed by quarto-dev/quarto-cli#14251 2.4.4 Link Purpose (In Context), 4.1.2 Name, Role, Value
tabindex :: #tabby-toggle_tabset (docs/presentations/revealjs/demo/index.html and examples/tabset.html only) 1 signature quarto-dev/quarto-cli#14970: tabby.js gives revealjs tabset tabs tabindex 1, 2, and so on, so Tab on an earlier slide moves focus to a tab on a hidden slide 2.4.3 Focus Order
landmark-no-duplicate-banner and landmark-unique on #quarto-header (docs/blog/posts/2026-05-07-new-blog-home/index.html only) 2 signatures quarto-dev/quarto-cli#14375: title-block-banner moves #title-block-header out of <main>, so the page has two top-level banners Best practice
region :: #quarto-embed-header (docs/authoring/penguins-preview.html and penguins-qmd.embed-preview.html only) 1 signature quarto-dev/quarto-cli#14968: the notebook preview header is a plain <div> outside every landmark Best practice
region :: .quarto-sidebar-toggle-title (docs/get-started/hello/positron.html and computations/positron.html only) 1 signature quarto-dev/quarto-cli#14969: the collapsed "On this page" toggle is a plain <div> that a keyboard cannot reach or open Best practice; the keyboard defect is a candidate failure of 2.1.1 Keyboard

The #cb entry came from the baseline of the old axe harness on feat/axe-a11y-harness. It was the only one of 27 old entries with a finding that still occurs and a note that is still true.

All 37 quarto-dev/quarto-cli#14378 entries clear when quarto-dev/quarto-cli#14816 ships. A scan of the 144 affected pages, rendered with quarto-cli main and then with #14816 merged on top, removed these 37 signatures and added no scrollable findings. The 36 new entries are limited to their pages because signatures such as scrollable-region-focusable :: pre also match revealjs slides, which #14816 does not change. Of the 10 scrollable signatures that #14816 does not clear, the 5 revealjs signatures are in the baseline under quarto-dev/quarto-cli#14817, and the Arquero notebook table is under third-party. The visually hidden code copies are not in the baseline.

The frame-title entries clear when quarto-dev/quarto-cli#14933 ships, because #2220 adds a code-preview-title to every preview. They are limited to the pages that have them because signatures such as frame-title :: iframe also match untitled video embeds, which quarto-web can fix.

The line-highlighting entries are limited to the two example decks because signatures such as color-contrast :: #bbbbbb on #ffffff can also match gray text in quarto-web content. These entries do not have a removal condition yet. quarto-dev/quarto-cli#14959 asks for documentation or a different highlight method, not a fix to the fade.

The two line-number signatures are one defect. axe adds #cb to the selector when a page has more than one code block, or copies of a block for stepped highlighting. The entries are limited to the seven decks because #cb > a[aria-label="*"] can also match line numbers on format: html pages, and a[aria-label="*"] can match any labeled link. They clear when quarto-dev/quarto-cli#14961 is fixed.

The quarto-dev/quarto-cli#14817 entries are limited to their decks because signatures such as scrollable-region-focusable :: body can also match a page in quarto-web content. body is 25 of the 30 instances: at 320x568, reveal.js switches to scroll view, and the whole page scrolls. The barrier is real, because a short viewport also brings back the slide scrollbar. The .latex code block did not reproduce in a local render, so this entry can disappear from a scan. The entries clear when quarto-dev/quarto-cli#14817 is fixed.

The two revealjs link-colour entries are limited to their decks because #2a76dd is also the default link colour of other revealjs content that quarto-web can override. They clear when quarto-dev/quarto-cli#14962 and quarto-dev/quarto-cli#14963 are fixed. axe flags both only at 320x568. At wider viewports reveal.js scales the slide, and axe reports the contrast as incomplete.

The slide-menu button entry applies to all pages, because Quarto adds the button to every revealjs deck, and the selector matches only this button. The bundled reveal.js-menu plugin makes the button. The upstream plugin is archived, so the fix goes into the copy in Quarto. The entry clears when quarto-dev/quarto-cli#14251 ships.

The page landmark entries are limited to their pages because #quarto-header is on every page, and region signatures can match content that quarto-web can fix. The banner entries cover the only rendered post with a title banner. The #quarto-embed-header entry also lists penguins-qmd.embed-preview.html, which Quarto renders only when .quarto/ does not exist (quarto-dev/quarto-cli#10756). The toggle entry shows only at 320x568, where a .column-page-right image on the two Positron pages covers the margin TOC. They clear when the matching quarto-cli issue is fixed.

The image-alt signatures are limited to the placeholder page because p > .img-fluid also matches content images on other pages that need alt text.

The tabindex entry is limited to the two decks with a tabset, because #tabby-toggle_tabset can also match a tabset on a format: html page. It clears when quarto-dev/quarto-cli#14970 is fixed.

Upstream: reveal.js (2 signatures)

Rule: landmark-complementary-is-top-level, an axe best practice.

reveal.js core renders the slide controls as <aside class="controls"> inside div.reveal[role=application], so the controls are a complementary landmark inside another landmark (hakimel/reveal.js#3369).

  • landmark-complementary-is-top-level :: .controls applies to all pages. The selector matches only the reveal.js controls.
  • landmark-complementary-is-top-level :: aside applies only to the 36 decks that have it. The signature can match any nested <aside>, so a real defect on another page still shows as new. A new deck shows this finding until its page is added to the entry.

Third-party: Observable Plot (20 signatures, 54 instances, all pages)

Rule: aria-prohibited-attr, 4.1.2 Name, Role, Value.

Plot puts an aria-label on the <g> of every mark and axis, with no role. There are 20 signatures because the same label occurs under different selector paths. The signatures are in two groups:

Third-party: Observable Inputs (7 signatures)

Rules: label, 4.1.2 Name, Role, Value, and empty-table-header (best practice).

  • Inputs.range sliders (3 signatures, 12 instances). Every call in quarto-web gives a label. But Inputs connects the <label for> to the number box only (range.js L57). The slider has no accessible name.
  • Inputs.table selection checkboxes (3 signatures, 48 instances). The row and header checkboxes have no label, and no option adds one (table.js L100-L101). Inputs 0.11.0 added select: false, which removes the checkboxes. But then the scroll container of the table has no focusable content, and axe reports scrollable-region-focusable (2.1.1 Keyboard).
  • The Inputs.table select-all header (1 signature, docs/interactive/ojs/code-reuse.html, data-sources.html, and examples/github.html only). The header cell holds only the unlabeled select-all checkbox, so axe also reports empty-table-header :: th. The entry is limited to these pages because th matches any empty header cell.

Scans with Inputs 0.11.0 and 0.12.0 gave the same 60 label instances. No issue in observablehq/inputs reports these defects.

The two .oi-3a86ea-input signatures are specific to Inputs, so they apply to all pages. When Quarto updates Inputs (quarto-dev/quarto-cli#14934), the class changes to .inputs-3a86ea-input and these two entries need the new selector. The other four signatures can match any unlabeled input, so they apply only to the pages that have them.

Third-party: leaflet (8 signatures)

  • aria-allowed-role and presentation-role-conflict on .leaflet-marker-icon (9 instances). leaflet.js gives each marker <img> role="button" and tabindex="0", but also alt="". These two rules are axe best practices, not WCAG failures. Three of the signatures are specific to the three maps on docs/authoring/article-layout.html. axe adds the htmlwidget ID of each map to these signatures, because the page has more than one map. The IDs are random, and they come from the frozen output of the page. When Quarto runs the code of the page again, the IDs change, and these three entries need the new IDs. Give empty table header cells an accessible name #2242 changed the IDs in this way.
  • color-contrast on the attribution control (2 signatures, 4 instances), 1.4.3 Contrast (Minimum). leaflet.css has no dark theme.
  • link-in-text-block :: a on the ESDIS link in the attribution of the map tiles (2 instances), 1.4.1 Use of Color. The link is #0078a8 on the #343a40 page text, 2.32:1. This entry applies only to docs/interactive/index.html and docs/interactive/widgets/jupyter.html. The signature drops the href, so it can also match any other plain link that has low contrast.

No leaflet issue matches these findings exactly.

Third-party: dygraphs (1 signature, 15 instances, docs/interactive/widgets/htmlwidgets.html only)

Rule: color-contrast :: #000000 on #151515, 1.4.3 Contrast (Minimum).

dygraph.js sets the color of the axis labels in an inline style, so CSS cannot change it. dygraph.js 2.0 moved the color into a stylesheet rule. But the R dygraphs package still bundles dygraph.js 1.1.1 (rstudio/dygraphs#241).

This entry applies to htmlwidgets.html only. The same signature also occurs on docs/authoring/penguins-preview.html, where it is a Quarto defect in the notebook cell labels.

Third-party: the @uwdata/arquero Observable notebook (1 signature, 2 instances, docs/interactive/ojs/examples/arquero.html only)

Rule: scrollable-region-focusable :: #ojs-cell > .observablehq > div, 2.1.1 Keyboard.

The example imports Arquero from the @uwdata/arquero Observable notebook, not from npm. The notebook adds a .view() method to tables. This method puts the table in a <div style="max-height: 270px; overflow: auto"> that has no tabindex, so a keyboard user cannot scroll it. The notebook is the correct place for the fix. The notebook is not in a repo, so no upstream issue is filed.

Third-party: vega-embed (1 signature, 4 instances on 4 pages)

Rule: summary-name :: summary, 4.1.2 Name, Role, Value.

vega-embed adds an actions menu to every Altair and Vega chart. The menu is a <details> element. Its <summary> contains only an icon SVG, so axe finds no name. The label ("Click to view actions") is a title on the <details>. VoiceOver reads this label, so no upstream issue is filed.

This entry applies only to the four pages with Altair charts: docs/authoring/notebook-embed.html, penguins-preview.html, penguins.html, and docs/presentations/revealjs/examples/executable-code-figure-size.html. The signature can also match any other <summary> that has no text.

Third-party: knitr and Arquero tables (6 signatures)

Four other empty-table-header signatures are not in the baseline. They come from tables written in quarto-web content, and #2242 fixes them.

False positive: the closed revealjs slide menu (1 signature, 1 instance, docs/presentations/revealjs/examples/executable-code-figure-size.html only)

Rule: color-contrast :: #aaaaaa on #ffffff, 1.4.3 Contrast (Minimum).

When the slide menu is closed, it is moved 300px to the left, and its overflow is hidden. This slide has no heading, so the menu uses the text of the slide as the title. The title is long, and about 12px of it shows at the left edge of the slide. axe calculates the contrast against the white slide. When the menu is open, the title is #aaaaaa on #333333, 5.44:1.

This entry applies to this deck only, because #aaaaaa on #ffffff can also match gray text in quarto-web content. A heading on the slide would give the menu a short title, and the entry could then be removed.

Intentional: slides without a title (1 signature, docs/presentations/revealjs/examples/background-no-title.html only)

Rule: empty-heading :: #section > h2, an axe best practice.

The Slide Backgrounds Without a Title example shows slides with only background attributes, so each slide renders an empty <h2>. The example stays as it is. quarto-dev/quarto-cli#14971 asks the docs to show a .visually-hidden title as the way to avoid the empty <h2>.

This entry applies to this deck only, because the signature can match an empty heading on any other deck.

* Seed _axe-baseline.json with the one carried-forward entry

Re-checked all 27 entries from the old harness baseline
(origin/feat/axe-a11y-harness:_tools/axe/baseline.json) against the
2026-09-15 quarto-web 1.11.4 scan. Only scrollable-region-focusable ::
#cb still has both a live signature and a note that still holds
(quarto-cli#14378, still open). Three entries were fixed upstream
(#14615, #14376) and no longer appear. The remaining 23 carried empty
notes with no recorded reasoning, so they're held open for a fresh
re-triage (accessibility repo strand ax-xmy5f5qm) rather than carried
forward blind.

* Accept the dygraphs axis-label contrast on the htmlwidgets page

dygraph.js sets the axis-label color with an inline style, so no CSS can
override it. dygraph.js >= 2.0 moves it into a stylesheet rule, but the R
dygraphs package still bundles 1.1.1 (rstudio/dygraphs#241). Scoped to
htmlwidgets.html: the same signature on penguins-preview.html is a real
Quarto defect that must stay visible.

* Accept the Observable Plot g[aria-label] findings as third-party

Plot puts an aria-label with no role on the <g> of every mark and axis.
20 signatures in two groups:

- 10 axis-tick and grid signatures are fixed in Plot 0.6.14
  (observablehq/plot#2018). Quarto bundles Plot 0.6.11, so these wait on
  quarto-cli#14934. Prune them when Quarto updates Plot.
- 10 data-mark signatures still occur in Plot 0.6.17. Plot labels marks
  on purpose (observablehq/plot#944); the open request for a role is
  observablehq/plot#1760.

* Accept the Observable Inputs unlabeled slider and checkbox findings as third-party

Inputs.range puts its <label for> on the number box only, so the slider
has no accessible name even when `label` is set. Inputs.table row and
header checkboxes have no label, and no option adds one. Inputs 0.12.0
has the same markup, and no upstream issue reports either defect.

The two .oi-3a86ea-input signatures are site-wide; the selector becomes
.inputs-3a86ea-input when Quarto updates Inputs (quarto-cli#14934). The
four generic signatures are scoped to their pages, so a new unlabeled
input elsewhere still shows as new.

* Accept the margin-citation aria-required-parent finding as upstream

quarto-cli#14755: Quarto relocates bibliography entries to the margin but
drops the role="list" wrapper they need, leaving role="listitem" orphaned.
1 instance on docs/authoring/article-layout.html.

* Accept the callout disclosure ARIA findings as upstream

quarto-cli#4934: collapsed callouts use a plain <div aria-expanded> instead
of the ARIA disclosure pattern (button + aria-controls), open since 2023.
3 signatures / 13 instances across brand, callouts, front-matter, the three
manuscript editor pages, github-pages, and the get-started editor pages.

* Accept the placeholder-shortcode alt findings as upstream

quarto-cli#14769: {{< placeholder >}} builds its <img> from pandoc.Image
with an always-empty caption, so no alt attribute reaches the output.
2 instances on docs/authoring/placeholder.html, scoped to that page --
the broader p > .img-fluid signature also covers real content images on
other pages that still need alt text of their own.

* Accept the leaflet marker-icon and attribution findings as third-party

leaflet.js sets role="button" and tabindex="0" on marker icons that also
carry alt="" (implicit presentation role), and its attribution control's
CSS isn't dark-mode aware. 7 signatures / 9 marker instances + 2 contrast
findings (2 instances each) across docs/interactive/index,
docs/interactive/widgets/{htmlwidgets,jupyter}, the revealjs demo, and the
three leaflet maps on docs/authoring/article-layout. No upstream issue —
this is leaflet's own markup and CSS, not Quarto's.

* Accept the knitr::kable() and Arquero table findings as third-party

knitr::kable() leaves the row-name corner cell as an empty <th> by
design, not oversight: yihui/knitr#1747, merged as PR #2500 on
2026-09-19, added scope="col"/scope="row" to kable's other header cells
but explicitly kept the corner cell blank and unscoped. 5 signatures / 7
instances across notebook-embed, penguins(-preview), three kable()
examples on article-layout, and the revealjs demo's Data tab.

Arquero's table viewer hardcodes background: #fff via inline JS style, so
its sticky header text fails contrast in dark mode (1 signature, 22
instances, reused on both the arquero example and libraries pages).

Each of the 5 kable entries repeats the full note, so the context
survives if any one entry is pruned.

* Accept the code-preview frame-title findings as upstream

The 35 code-preview iframes have no title (quarto-cli#14770). They clear
when quarto-cli#14933 ships, because fix/code-preview-titles already sets
code-preview-title on every preview.

* Accept the revealjs meta-viewport finding as upstream

* Accept the scrollable-region findings that quarto-cli#14816 fixes as upstream

* Accept the revealjs line-highlight fade findings as upstream

* Accept the revealjs line-number anchor findings as upstream

* Accept the vega-embed actions-menu summary-name findings as third-party

* Accept the Leaflet ESDIS attribution link findings as third-party

* Accept the Arquero notebook table-view scrollable-region finding as third-party

* Accept the revealjs scrollable-region findings as upstream quarto-cli#14817

* Accept the revealjs link and tab contrast findings as upstream quarto-cli#14962 and #14963

* Accept the revealjs closed slide-menu contrast finding as a false positive

* Accept the revealjs slide-menu button link-name finding as upstream quarto-cli#14249

* Accept the page landmark findings as upstream quarto-cli#14375, #14968 and #14969

* Accept the revealjs tabindex, nested controls landmark, and Inputs.table header findings

Upstream quarto-cli#14970 and hakimel/reveal.js#3369, and third-party Observable Inputs.

* Accept the background-no-title empty-heading finding as an intentional example

* Accept the embed-header region finding on the second notebook preview page

* Update the article-layout leaflet baseline entries to the IDs refrozen by #2242

The three presentation-role-conflict signatures include the htmlwidget ID of each map. #2242 re-executed docs/authoring/article-layout.qmd, so the frozen IDs changed.

* Tidy baseline notes, group entries by owner, and limit .collapsed to its pages

Notes now read '<type>: <issue-ref>. <description>.' with full org/repo
references and no commentary or links. Entries follow the PR description's
order. aria-allowed-attr :: .collapsed is limited to the four get-started
authoring pages, because .collapsed is the generic Bootstrap class.

* Document the axe baseline note format in the README and a Claude rule

(cherry picked from commit a4b9a81)
@cderv

cderv commented Oct 2, 2026

Copy link
Copy Markdown
Member

/deploy-preview

@github-actions

github-actions Bot commented Oct 2, 2026

Copy link
Copy Markdown
Contributor Author

📝 Preview Deployment

🔍 Full site preview: https://deploy-preview-2260.quarto.org

@cwickham
cwickham merged commit 9fddbfc into prerelease Oct 2, 2026
@cwickham
cwickham deleted the sync-2208-to-prerelease branch October 2, 2026 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants