Repository navigation
[Sync to prerelease] Add an axe baseline for known third-party and upstream findings - #2260
Merged
Merged
Conversation
* Seed _axe-baseline.json with the one carried-forward entry Re-checked all 27 entries from the old harness baseline (origin/feat/axe-a11y-harness:_tools/axe/baseline.json) against the 2026-09-15 quarto-web 1.11.4 scan. Only scrollable-region-focusable :: #cb still has both a live signature and a note that still holds (quarto-cli#14378, still open). Three entries were fixed upstream (#14615, #14376) and no longer appear. The remaining 23 carried empty notes with no recorded reasoning, so they're held open for a fresh re-triage (accessibility repo strand ax-xmy5f5qm) rather than carried forward blind. * Accept the dygraphs axis-label contrast on the htmlwidgets page dygraph.js sets the axis-label color with an inline style, so no CSS can override it. dygraph.js >= 2.0 moves it into a stylesheet rule, but the R dygraphs package still bundles 1.1.1 (rstudio/dygraphs#241). Scoped to htmlwidgets.html: the same signature on penguins-preview.html is a real Quarto defect that must stay visible. * Accept the Observable Plot g[aria-label] findings as third-party Plot puts an aria-label with no role on the <g> of every mark and axis. 20 signatures in two groups: - 10 axis-tick and grid signatures are fixed in Plot 0.6.14 (observablehq/plot#2018). Quarto bundles Plot 0.6.11, so these wait on quarto-cli#14934. Prune them when Quarto updates Plot. - 10 data-mark signatures still occur in Plot 0.6.17. Plot labels marks on purpose (observablehq/plot#944); the open request for a role is observablehq/plot#1760. * Accept the Observable Inputs unlabeled slider and checkbox findings as third-party Inputs.range puts its <label for> on the number box only, so the slider has no accessible name even when `label` is set. Inputs.table row and header checkboxes have no label, and no option adds one. Inputs 0.12.0 has the same markup, and no upstream issue reports either defect. The two .oi-3a86ea-input signatures are site-wide; the selector becomes .inputs-3a86ea-input when Quarto updates Inputs (quarto-cli#14934). The four generic signatures are scoped to their pages, so a new unlabeled input elsewhere still shows as new. * Accept the margin-citation aria-required-parent finding as upstream quarto-cli#14755: Quarto relocates bibliography entries to the margin but drops the role="list" wrapper they need, leaving role="listitem" orphaned. 1 instance on docs/authoring/article-layout.html. * Accept the callout disclosure ARIA findings as upstream quarto-cli#4934: collapsed callouts use a plain <div aria-expanded> instead of the ARIA disclosure pattern (button + aria-controls), open since 2023. 3 signatures / 13 instances across brand, callouts, front-matter, the three manuscript editor pages, github-pages, and the get-started editor pages. * Accept the placeholder-shortcode alt findings as upstream quarto-cli#14769: {{< placeholder >}} builds its <img> from pandoc.Image with an always-empty caption, so no alt attribute reaches the output. 2 instances on docs/authoring/placeholder.html, scoped to that page -- the broader p > .img-fluid signature also covers real content images on other pages that still need alt text of their own. * Accept the leaflet marker-icon and attribution findings as third-party leaflet.js sets role="button" and tabindex="0" on marker icons that also carry alt="" (implicit presentation role), and its attribution control's CSS isn't dark-mode aware. 7 signatures / 9 marker instances + 2 contrast findings (2 instances each) across docs/interactive/index, docs/interactive/widgets/{htmlwidgets,jupyter}, the revealjs demo, and the three leaflet maps on docs/authoring/article-layout. No upstream issue — this is leaflet's own markup and CSS, not Quarto's. * Accept the knitr::kable() and Arquero table findings as third-party knitr::kable() leaves the row-name corner cell as an empty <th> by design, not oversight: yihui/knitr#1747, merged as PR #2500 on 2026-09-19, added scope="col"/scope="row" to kable's other header cells but explicitly kept the corner cell blank and unscoped. 5 signatures / 7 instances across notebook-embed, penguins(-preview), three kable() examples on article-layout, and the revealjs demo's Data tab. Arquero's table viewer hardcodes background: #fff via inline JS style, so its sticky header text fails contrast in dark mode (1 signature, 22 instances, reused on both the arquero example and libraries pages). Each of the 5 kable entries repeats the full note, so the context survives if any one entry is pruned. * Accept the code-preview frame-title findings as upstream The 35 code-preview iframes have no title (quarto-cli#14770). They clear when quarto-cli#14933 ships, because fix/code-preview-titles already sets code-preview-title on every preview. * Accept the revealjs meta-viewport finding as upstream * Accept the scrollable-region findings that quarto-cli#14816 fixes as upstream * Accept the revealjs line-highlight fade findings as upstream * Accept the revealjs line-number anchor findings as upstream * Accept the vega-embed actions-menu summary-name findings as third-party * Accept the Leaflet ESDIS attribution link findings as third-party * Accept the Arquero notebook table-view scrollable-region finding as third-party * Accept the revealjs scrollable-region findings as upstream quarto-cli#14817 * Accept the revealjs link and tab contrast findings as upstream quarto-cli#14962 and #14963 * Accept the revealjs closed slide-menu contrast finding as a false positive * Accept the revealjs slide-menu button link-name finding as upstream quarto-cli#14249 * Accept the page landmark findings as upstream quarto-cli#14375, #14968 and #14969 * Accept the revealjs tabindex, nested controls landmark, and Inputs.table header findings Upstream quarto-cli#14970 and hakimel/reveal.js#3369, and third-party Observable Inputs. * Accept the background-no-title empty-heading finding as an intentional example * Accept the embed-header region finding on the second notebook preview page * Update the article-layout leaflet baseline entries to the IDs refrozen by #2242 The three presentation-role-conflict signatures include the htmlwidget ID of each map. #2242 re-executed docs/authoring/article-layout.qmd, so the frozen IDs changed. * Tidy baseline notes, group entries by owner, and limit .collapsed to its pages Notes now read '<type>: <issue-ref>. <description>.' with full org/repo references and no commentary or links. Entries follow the PR description's order. aria-allowed-attr :: .collapsed is limited to the four get-started authoring pages, because .collapsed is the generic Bootstrap class. * Document the axe baseline note format in the README and a Claude rule (cherry picked from commit a4b9a81)
Member
|
/deploy-preview |
Contributor
Author
📝 Preview Deployment🔍 Full site preview: https://deploy-preview-2260.quarto.org |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Sync of #2208 to
prerelease.Original PR
Add an axe baseline for known third-party and upstream findings
Summary
This PR adds
_axe-baseline.json, the baseline file forquarto call axe(quarto-dev/quarto-cli#14815). A scan compares its findings with the baseline. It reports only the findings that are not in the baseline as new.The baseline has 128 signatures. Each one is a finding that quarto-web cannot fix in its own content, a false positive, or an intentional example:
third-party:the markup comes from a library, for example Observable Plot or leaflet.upstream:the markup comes from Quarto, and a quarto-cli issue tracks the fix.false-positive:axe reports a defect that a user does not see.intentional:a docs example shows the defect on purpose.Each entry has a
notethat gives the cause, the owner, and the condition to remove the entry. Each note is complete by itself, so the context stays if another entry is removed.Most entries apply to all pages. An entry is limited to specific pages when its signature can also match a real defect in quarto-web content. Then a new defect of that type on a different page still shows as new.
Entries
Upstream: Quarto
scrollable-region-focusable :: #cb(all pages)tabindex. Fixed by quarto-dev/quarto-cli#14816scrollable-region-focusablecode blocks and cell outputs (only the pages that have them)aria-allowed-attron collapsed callout headers (all pages;.collapsedonly on the fourdocs/get-started/authoring/pages)<div aria-expanded>, not the disclosure patternimage-alton{{< placeholder >}}images (docs/authoring/placeholder.htmlonly)<img>with noaltaria-required-parent :: #ref-xie(docs/authoring/article-layout.htmlonly)role="listitem", but Quarto removes theirrole="list"parentframe-titleoncode-previewiframes (docs/presentations/revealjs/index.html,advanced.html, anddocs/authoring/code-annotation.htmlonly)code-previewmakes an<iframe>with notitle. Fixed by quarto-dev/quarto-cli#14933meta-viewport :: meta(all pages)maximum-scale=1.0, user-scalable=noon every deckcolor-contraston faded lines in revealjs line highlighting (docs/presentations/revealjs/examples/line-highlighting-1.htmlandline-highlighting-2.htmlonly)code-line-numbersfades the other lines toopacity: 0.4. No opacity value meets 4.5:1 and still shows the highlightaria-prohibited-attron revealjs code line numbers (docs/presentations/revealjs/demo/index.html,demo/mini/auto-animate-code.html,examples/code-echo.html, andexamples/line-highlighting-1.htmltoline-highlighting-4.htmlonly)hreffrom line-number links, so theiraria-labelis on an<a>that is not a linkscrollable-region-focusableon revealjs slides, the slide menu, and a code block (body,#slide,#scrollable-slide,.active-menu-panel,.latex; 26 revealjs decks only)overflow-y: auto, and the arrow keys move between slides instead of scrolling, so keyboard users cannot reach overflowing contentcolor-contrast :: #2a76dd on #ffffff(docs/presentations/revealjs/demo/index.htmlonly)$link-coloris 4.44:1 on white, which fails for small text such as the footercolor-contrast :: #2a76dd on #bbbbbb(docs/presentations/revealjs/examples/tabset.htmlonly)#bbbbbbfill, so its text is 2.31:1link-name :: .slide-menu-button > a(all pages)<a href="#">with only an icon, so it has no accessible name. Fixed by quarto-dev/quarto-cli#14251tabindex :: #tabby-toggle_tabset(docs/presentations/revealjs/demo/index.htmlandexamples/tabset.htmlonly)tabindex1, 2, and so on, so Tab on an earlier slide moves focus to a tab on a hidden slidelandmark-no-duplicate-bannerandlandmark-uniqueon#quarto-header(docs/blog/posts/2026-05-07-new-blog-home/index.htmlonly)title-block-bannermoves#title-block-headerout of<main>, so the page has two top-level bannersregion :: #quarto-embed-header(docs/authoring/penguins-preview.htmlandpenguins-qmd.embed-preview.htmlonly)<div>outside every landmarkregion :: .quarto-sidebar-toggle-title(docs/get-started/hello/positron.htmlandcomputations/positron.htmlonly)<div>that a keyboard cannot reach or openThe
#cbentry came from the baseline of the old axe harness onfeat/axe-a11y-harness. It was the only one of 27 old entries with a finding that still occurs and a note that is still true.All 37 quarto-dev/quarto-cli#14378 entries clear when quarto-dev/quarto-cli#14816 ships. A scan of the 144 affected pages, rendered with quarto-cli
mainand then with #14816 merged on top, removed these 37 signatures and added no scrollable findings. The 36 new entries are limited to their pages because signatures such asscrollable-region-focusable :: prealso match revealjs slides, which #14816 does not change. Of the 10 scrollable signatures that #14816 does not clear, the 5 revealjs signatures are in the baseline under quarto-dev/quarto-cli#14817, and the Arquero notebook table is under third-party. The visually hidden code copies are not in the baseline.The
frame-titleentries clear when quarto-dev/quarto-cli#14933 ships, because #2220 adds acode-preview-titleto every preview. They are limited to the pages that have them because signatures such asframe-title :: iframealso match untitled video embeds, which quarto-web can fix.The line-highlighting entries are limited to the two example decks because signatures such as
color-contrast :: #bbbbbb on #ffffffcan also match gray text in quarto-web content. These entries do not have a removal condition yet. quarto-dev/quarto-cli#14959 asks for documentation or a different highlight method, not a fix to the fade.The two line-number signatures are one defect. axe adds
#cbto the selector when a page has more than one code block, or copies of a block for stepped highlighting. The entries are limited to the seven decks because#cb > a[aria-label="*"]can also match line numbers onformat: htmlpages, anda[aria-label="*"]can match any labeled link. They clear when quarto-dev/quarto-cli#14961 is fixed.The quarto-dev/quarto-cli#14817 entries are limited to their decks because signatures such as
scrollable-region-focusable :: bodycan also match a page in quarto-web content.bodyis 25 of the 30 instances: at 320x568, reveal.js switches to scroll view, and the whole page scrolls. The barrier is real, because a short viewport also brings back the slide scrollbar. The.latexcode block did not reproduce in a local render, so this entry can disappear from a scan. The entries clear when quarto-dev/quarto-cli#14817 is fixed.The two revealjs link-colour entries are limited to their decks because
#2a76ddis also the default link colour of other revealjs content that quarto-web can override. They clear when quarto-dev/quarto-cli#14962 and quarto-dev/quarto-cli#14963 are fixed. axe flags both only at 320x568. At wider viewports reveal.js scales the slide, and axe reports the contrast as incomplete.The slide-menu button entry applies to all pages, because Quarto adds the button to every revealjs deck, and the selector matches only this button. The bundled reveal.js-menu plugin makes the button. The upstream plugin is archived, so the fix goes into the copy in Quarto. The entry clears when quarto-dev/quarto-cli#14251 ships.
The page landmark entries are limited to their pages because
#quarto-headeris on every page, andregionsignatures can match content that quarto-web can fix. The banner entries cover the only rendered post with a title banner. The#quarto-embed-headerentry also listspenguins-qmd.embed-preview.html, which Quarto renders only when.quarto/does not exist (quarto-dev/quarto-cli#10756). The toggle entry shows only at 320x568, where a.column-page-rightimage on the two Positron pages covers the margin TOC. They clear when the matching quarto-cli issue is fixed.The
image-altsignatures are limited to the placeholder page becausep > .img-fluidalso matches content images on other pages that need alt text.The
tabindexentry is limited to the two decks with a tabset, because#tabby-toggle_tabsetcan also match a tabset on aformat: htmlpage. It clears when quarto-dev/quarto-cli#14970 is fixed.Upstream: reveal.js (2 signatures)
Rule:
landmark-complementary-is-top-level, an axe best practice.reveal.js core renders the slide controls as
<aside class="controls">insidediv.reveal[role=application], so the controls are a complementary landmark inside another landmark (hakimel/reveal.js#3369).landmark-complementary-is-top-level :: .controlsapplies to all pages. The selector matches only the reveal.js controls.landmark-complementary-is-top-level :: asideapplies only to the 36 decks that have it. The signature can match any nested<aside>, so a real defect on another page still shows as new. A new deck shows this finding until its page is added to the entry.Third-party: Observable Plot (20 signatures, 54 instances, all pages)
Rule:
aria-prohibited-attr, 4.1.2 Name, Role, Value.Plot puts an
aria-labelon the<g>of every mark and axis, with norole. There are 20 signatures because the same label occurs under different selector paths. The signatures are in two groups:rect,frame,line,rule,tick. These still occur in Plot 0.6.17. Plot labels marks on purpose (ariaLabel, ariaDescription, ariaHidden observablehq/plot#710, [a11y] Accessibility issue with plot rect observablehq/plot#944). The open request for a role is SVGGElements without text content are not meaningful to user agents observablehq/plot#1760.Third-party: Observable Inputs (7 signatures)
Rules:
label, 4.1.2 Name, Role, Value, andempty-table-header(best practice).Inputs.rangesliders (3 signatures, 12 instances). Every call in quarto-web gives alabel. But Inputs connects the<label for>to the number box only (range.jsL57). The slider has no accessible name.Inputs.tableselection checkboxes (3 signatures, 48 instances). The row and header checkboxes have no label, and no option adds one (table.jsL100-L101). Inputs 0.11.0 addedselect: false, which removes the checkboxes. But then the scroll container of the table has no focusable content, and axe reportsscrollable-region-focusable(2.1.1 Keyboard).Inputs.tableselect-all header (1 signature,docs/interactive/ojs/code-reuse.html,data-sources.html, andexamples/github.htmlonly). The header cell holds only the unlabeled select-all checkbox, so axe also reportsempty-table-header :: th. The entry is limited to these pages becausethmatches any empty header cell.Scans with Inputs 0.11.0 and 0.12.0 gave the same 60
labelinstances. No issue inobservablehq/inputsreports these defects.The two
.oi-3a86ea-inputsignatures are specific to Inputs, so they apply to all pages. When Quarto updates Inputs (quarto-dev/quarto-cli#14934), the class changes to.inputs-3a86ea-inputand these two entries need the new selector. The other four signatures can match any unlabeled input, so they apply only to the pages that have them.Third-party: leaflet (8 signatures)
aria-allowed-roleandpresentation-role-conflicton.leaflet-marker-icon(9 instances). leaflet.js gives each marker<img>role="button"andtabindex="0", but alsoalt="". These two rules are axe best practices, not WCAG failures. Three of the signatures are specific to the three maps ondocs/authoring/article-layout.html. axe adds the htmlwidget ID of each map to these signatures, because the page has more than one map. The IDs are random, and they come from the frozen output of the page. When Quarto runs the code of the page again, the IDs change, and these three entries need the new IDs. Give empty table header cells an accessible name #2242 changed the IDs in this way.color-contraston the attribution control (2 signatures, 4 instances), 1.4.3 Contrast (Minimum).leaflet.csshas no dark theme.link-in-text-block :: aon the ESDIS link in the attribution of the map tiles (2 instances), 1.4.1 Use of Color. The link is#0078a8on the#343a40page text, 2.32:1. This entry applies only todocs/interactive/index.htmlanddocs/interactive/widgets/jupyter.html. The signature drops thehref, so it can also match any other plain link that has low contrast.No leaflet issue matches these findings exactly.
Third-party: dygraphs (1 signature, 15 instances,
docs/interactive/widgets/htmlwidgets.htmlonly)Rule:
color-contrast :: #000000 on #151515, 1.4.3 Contrast (Minimum).dygraph.js sets the color of the axis labels in an inline style, so CSS cannot change it. dygraph.js 2.0 moved the color into a stylesheet rule. But the R
dygraphspackage still bundles dygraph.js 1.1.1 (rstudio/dygraphs#241).This entry applies to
htmlwidgets.htmlonly. The same signature also occurs ondocs/authoring/penguins-preview.html, where it is a Quarto defect in the notebook cell labels.Third-party: the
@uwdata/arqueroObservable notebook (1 signature, 2 instances,docs/interactive/ojs/examples/arquero.htmlonly)Rule:
scrollable-region-focusable :: #ojs-cell > .observablehq > div, 2.1.1 Keyboard.The example imports Arquero from the
@uwdata/arqueroObservable notebook, not from npm. The notebook adds a.view()method to tables. This method puts the table in a<div style="max-height: 270px; overflow: auto">that has notabindex, so a keyboard user cannot scroll it. The notebook is the correct place for the fix. The notebook is not in a repo, so no upstream issue is filed.Third-party: vega-embed (1 signature, 4 instances on 4 pages)
Rule:
summary-name :: summary, 4.1.2 Name, Role, Value.vega-embed adds an actions menu to every Altair and Vega chart. The menu is a
<details>element. Its<summary>contains only an icon SVG, so axe finds no name. The label ("Click to view actions") is atitleon the<details>. VoiceOver reads this label, so no upstream issue is filed.This entry applies only to the four pages with Altair charts:
docs/authoring/notebook-embed.html,penguins-preview.html,penguins.html, anddocs/presentations/revealjs/examples/executable-code-figure-size.html. The signature can also match any other<summary>that has no text.Third-party: knitr and Arquero tables (6 signatures)
empty-table-headeronknitr::kable()output (5 signatures, 7 instances). This rule is an axe best practice.kable()leaves the corner cell above the row names as an empty<th>on purpose. feature request: more accessibility/navigation tags in kable_html yihui/knitr#1747, merged as kable(): accessibility scope attributes in HTML tables (#1747) yihui/knitr#2500, addedscopeto the other header cells but kept the corner cell empty.color-contrast :: #f8f8f8 on #ffffffin the Arquero table viewer (1 signature, 22 instances), 1.4.3 Contrast (Minimum). Arquero setsbackground: #fffon the sticky header in an inline style. In dark mode, the header text gets the light text color of the theme.Four other
empty-table-headersignatures are not in the baseline. They come from tables written in quarto-web content, and #2242 fixes them.False positive: the closed revealjs slide menu (1 signature, 1 instance,
docs/presentations/revealjs/examples/executable-code-figure-size.htmlonly)Rule:
color-contrast :: #aaaaaa on #ffffff, 1.4.3 Contrast (Minimum).When the slide menu is closed, it is moved 300px to the left, and its overflow is hidden. This slide has no heading, so the menu uses the text of the slide as the title. The title is long, and about 12px of it shows at the left edge of the slide. axe calculates the contrast against the white slide. When the menu is open, the title is
#aaaaaaon#333333, 5.44:1.This entry applies to this deck only, because
#aaaaaaon#ffffffcan also match gray text in quarto-web content. A heading on the slide would give the menu a short title, and the entry could then be removed.Intentional: slides without a title (1 signature,
docs/presentations/revealjs/examples/background-no-title.htmlonly)Rule:
empty-heading :: #section > h2, an axe best practice.The Slide Backgrounds Without a Title example shows slides with only background attributes, so each slide renders an empty
<h2>. The example stays as it is. quarto-dev/quarto-cli#14971 asks the docs to show a.visually-hiddentitle as the way to avoid the empty<h2>.This entry applies to this deck only, because the signature can match an empty heading on any other deck.