Error field is not identified using color differences only #1914

giacomo-petri · 2022-09-05T08:34:11Z

Proposal for a new rule to check errors are not identified using color differences only.

Closes issue(s):

new rule

Need for Call for Review:
This will require a 2 weeks Call for Review

Pull Request Etiquette

When creating PR:

Make sure you're requesting to pull a branch (right side) to the develop branch (left side).
Make sure you do not remove the "How to Review and Approve" section in your pull request description

After creating PR:

Add yourself (and co-authors) as "Assignees" for PR.
Add label to indicate if it's a Rule, Definition or Chore.
Link the PR to any issue it solves. This will be done automatically by referencing the issue at the top of this comment in the indicated place.
Optionally request feedback from anyone in particular by assigning them as "Reviewers".

When merging a PR:

Close any issue that the PR resolves. This will happen automatically upon merging if the PR was correctly linked to the issue, e.g. by referencing the issue at the top of this comment.

How to Review And Approve

Go to the “Files changed” tab
Here you will have the option to leave comments on different lines.
Once the review is completed, find the “Review changes” button in the top right, select “Approve” (if you are really confident in the rule) or "Request changes" and click “Submit review”.
Make sure to also review the proposed Call for Review period. In case of disagreement, the longer period wins.

Jym77

Good work. There are still quite a bunch of precision to add to the Applicability and Expectation to meet the requirements of ACT rules 😃

_rules/error-field-is-not-identified-using-color-only

Jym77 · 2022-09-08T07:58:26Z

_rules/error-field-is-not-identified-using-color-only

+
+## Applicability
+
+This rule applies to any error field that is [visible][].


What is an "error field"? This needs an unambiguous and objective definition.

_rules/error-field-is-not-identified-using-color-only

Jym77 · 2022-09-08T08:30:39Z

_rules/error-field-is-not-identified-using-color-only

+
+```html
+<label for="first_name" style="color:#dd0000">First Name</label>
+<input type="text" id="first_name" autocomplete="given-name" style="border:3px solid #dd0000" aria-invalid="true">


Making it required would make it more obvious the reason why this is an error, especially with an explicitly optional field in the form. Makin the last name also required could make sense.

same as above

Jym77 · 2022-09-08T08:39:07Z

_rules/error-field-is-not-identified-using-color-only

+
+### Passed
+
+#### Passed Example 1


I feel we need examples that use something else than text inputs and other errors than absence of a required field. That can be both other built-in errors (check the list of constraint validation errors for inspiration); or custom errors (e.g. "password must contain at least 8 characters, …" that is usually checked by some script).

Notably, that would show that the rule also considers custom errors.
Also, I do believe that if built-in constraint validations are failing, the UA should automagically set aria-invalid (needs to be tested).

Updated with different examples and added more details in what's failing.
See also my previous feedback about "required" and browser form validation (something to add in assumptions).

Jym77 · 2022-09-08T08:40:21Z

_rules/error-field-is-not-identified-using-color-only

+<label for="first_name" style="color:#dd0000">First Name</label>
+<input type="text" id="first_name" autocomplete="given-name" style="border:1px solid #dd0000" aria-invalid="true">
+<label for="last_name" style="color:#333333">Last Name</label>
+<input type="text" id="last_name" autocomplete="family-name" style="border:1px solid #b0b0b0" value="Doe">


Could be an example where the color difference is only on the field (the labels don't even have a color difference), and one where it is only on the label.

updated keeping both label and border colors and added 2 failed examples for only label color and only input border color

Added also inapplicable example with hidden form.

Jym77 · 2022-09-08T08:40:57Z

_rules/error-field-is-not-identified-using-color-only

+
+### Inapplicable
+
+#### Inapplicable Example 1


There should be an example where there are only non-errored form fields.

Jym77 · 2022-09-08T08:43:49Z

_rules/error-field-is-not-identified-using-color-only

+
+```html
+<p>This is a paragraph.</p>
+```


You need to add the reference list of the terms used in the rule.
For example, you use [visible][], so there should be here a definition:

[visible]: #visible 'Definition of Visible'

(see how it's done in other rules files

…is-not-identified-using-color-only-i3z4v8.md

Jym77

We need to define "visual context" (or find a way to do without it...)

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

Jym77 · 2022-10-06T11:05:21Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+The email `input` value is missing the "@" symbol. Its visual error indicator (the first couple of `label` and `input` elements) conveyed the presence of an error through its CSS properties:
+- lightness (difference in relative luminance between the colors leads to a contrast ratio greater than 3:1) AND;
+- font-weight: bold.


Suggested change

The email `input` value is missing the "@" symbol. Its visual error indicator (the first couple of `label` and `input` elements) conveyed the presence of an error through its CSS properties:

- lightness (difference in relative luminance between the colors leads to a contrast ratio greater than 3:1) AND;

- font-weight: bold.

The email `input` value is missing the "@" symbol. Its visual error indicator (the first couple of `label` and `input` elements) conveyed the presence of an error through its CSS properties:

- lightness (difference in relative luminance between the colors leads to a contrast ratio greater than 3:1); and

- `font-weight: bold`.

I think it would make sense to have the different contrast being the only difference. Otherwise, tools or methodologies that ignore it but accept font-weight would still be considered consistent.

I was trying to maintain best practices to the passed example. Do you think could be acceptable to remove font-weight: bold from the passed example description, but keep it in the passed example code as per new edits?

We also need to exert the rule to its limit (and the tools). Since the rule considers that a single difference is enough, we should have test cases with that, even if they are not best practice. This is part of the One thing at a time guideline in our design guide.
(actually, the case could even has a contrast ratio of exactly 3:1 to ensure that tools do accept that limit)
We can add to the background that while contrast alone is enough, it is a better practice to also use something else.

font-weight removed and added a paragraph in background section.

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

Jym77 · 2022-10-06T14:35:58Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+## Applicability
+
+This rule applies to any [visible][] [input error][] together with its visual context.


I'm not sure that "visual context" is objective enough for Applicability 🤔
(see the discussions we have on the "link context" definition)

@Jym77,

I've removed the generic "visual context" with a more detailed list of elements that should be included in the evaluation.
I'm not sure about the "closest error message that refers to it" is objective enough.

Co-authored-by: Jean-Yves Moyen <jym@siteimprove.com>

Jym77 · 2023-01-19T12:14:48Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+This rule applies to any [visible][] [input error][] together with:
+- its [programmatic label]; and
+- the closest error message that refers to it.


Indeed, "closest" is ambiguous. It could notably be distance in the DOM tree, or physical distance on screen between the painted elements.

Jym77 · 2023-01-19T12:16:25Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+This rule applies to any [visible][] [input error][] together with:
+- its [programmatic label]; and
+- the closest error message that refers to it.


I'm also not sure that "error message" is unambiguous.
For example, putting a red cross next to errored field and a green check mark next to good ones, is a red cross an "error message" (it's not really a message)? (or the red border of Passed Example 3)

Jym77 · 2023-01-19T12:16:51Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+## Expectation
+
+Each test target, together with its visual context, identify the presence of errors not only through color differences.


We need to change "visual context" here also.

Jym77 · 2023-01-19T12:19:15Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+## Applicability
+
+This rule applies to any [visible][] [input error][] together with:


Not fully convinced that "input error" is unambiguous enough.

information provided by the user that is not accepted

Does that mean that the field in which it was inputed is put in a validation error state? Or just that some code checked it and rejected it, without changing the validation state? Actually, this refers to the information, not even to the form field, is it what you want the rule to target?

Jym77 · 2023-01-19T12:24:29Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+- [Understanding Success Criterion 1.4.1: Use of Color](https://www.w3.org/WAI/WCAG21/Understanding/use-of-color.html)
+- [F81 - Failure of Success Criterion 1.4.1 due to identifying required or error fields using color differences only](https://www.w3.org/WAI/WCAG21/Techniques/failures/F81.html)
+
+It is possible for an error field to be distinguishable from adjacent fields but still not have a meaningful error message. In that case, it would pass Success Criterion 1.4.1: Use of Color but still fail [Success Criterion 3.3.1: Error Identification](https://www.w3.org/WAI/WCAG21/Understanding/error-identification.html) and, if applicable, [Scuccess Criterion 3.3.3: Error Suggestion](https://www.w3.org/WAI/WCAG21/Understanding/error-suggestion.html).


Suggested change

It is possible for an error field to be distinguishable from adjacent fields but still not have a meaningful error message. In that case, it would pass Success Criterion 1.4.1: Use of Color but still fail [Success Criterion 3.3.1: Error Identification](https://www.w3.org/WAI/WCAG21/Understanding/error-identification.html) and, if applicable, [Scuccess Criterion 3.3.3: Error Suggestion](https://www.w3.org/WAI/WCAG21/Understanding/error-suggestion.html).

It is possible for an error field to be distinguishable from adjacent fields but still not have a meaningful error message. In that case, it would pass Success Criterion 1.4.1: Use of Color but still fail [Success Criterion 3.3.1: Error Identification](https://www.w3.org/WAI/WCAG21/Understanding/error-identification.html) and, if applicable, [Success Criterion 3.3.3: Error Suggestion](https://www.w3.org/WAI/WCAG21/Understanding/error-suggestion.html).

Jym77 · 2023-01-19T12:25:53Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+#### Passed Example 2
+
+The "phone number" `input` value doesn't match the `type` value. Its visual error indicator (the first couple of `label` and `input` elements) conveys the presence of an error through its CSS properties:


This description uses "visual error indicator", which doesn't match the "error message" used in the Applicability. What does "visual error indicator" refers to?
(same on other descriptions)

Jym77 · 2023-01-19T12:28:43Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+#### Failed Example 1
+
+The "first name" `input` value is empty. Its visual error indicator (the red border color of the first `input` element) conveys the presence of an error through color (hue) differences only.


Maybe add the contrast ratio with the other border color, to drive the nail in.

Jym77 · 2023-01-19T12:29:44Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+#### Inapplicable Example 1
+
+There are no error fields.


These descriptions should talk about "input error", not "error field", in order to match the Applicability.

…dentified-using-color-only

I've completely revised the rule by setting objective applicability and unambiguous expectation that should address all the remaining items you've listed above

Jym77

I think we need to find a way to tell in the Expectation what we are comparing the errored field with.

Jym77 · 2023-05-25T09:52:29Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+## Applicability
+
+This rule applies to each [HTML element][] that is [visible][] and has one of the following [semantic roles][]:


Suggested change

This rule applies to each [HTML element][] that is [visible][] and has one of the following [semantic roles][]:

This rule applies to any [HTML element][] that is [visible][] and has one of the following [semantic roles][]:

Jym77 · 2023-05-25T09:54:56Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+Each test target either has no [visible][] [form field error indicators][], or at least one of the [visible][] [form field error indicators][] identifies the presence of errors not only through color differences.
+
+## Assumptions
+
+When content is communicated using colors that vary not just in hue but also have a substantial difference in brightness, this constitutes an extra visual differentiation provided that the difference in the colors' relative luminance creates a contrast ratio of 3:1 or more.


I think we somehow need to put in the Expectation the fact that "contrast of 3:1 or more" is not a color difference.
We probably need to keep an assumption that this is indeed enough, but we also want to say in the "normative" part that the rule accepts this.

Jym77 · 2023-05-25T10:02:13Z

_rules/error-field-is-not-identified-using-color-only-i3z4v8.md

+
+## Expectation
+
+Each test target either has no [visible][] [form field error indicators][], or at least one of the [visible][] [form field error indicators][] identifies the presence of errors not only through color differences.


Seeing a potential pitfall here (also when looking at examples):
This somewhat implicitly consider that we have something to compare the error indicator to, in order to decide whether it is color differences or more (i.e. "differences" with what?)

This could either be another field, not in error, in the same form, or the same field when not errored.
The second solution somewhat requires a statefull rule and the tester to act (and put the field in error). This hopefully will be doable soonish (once we've solved #1953 / #2045), but that may still take time.
The first point could be doable by targeting pairs of field, passing if they are both errored or none errored, and comparing if only one is errored 🤔

Put otherwise: if there is only one form field in the page, and it is in error, how do we check the rule?

This could either be another field, not in error, in the same form

Comparing a pair of inputs, where one fails and the other does not, is not an optimal approach. Additionally, inputs may have different styles, such as checkboxes or text fields. Therefore, it is advisable to only compare inputs of the same type and with the same CSS.

Put otherwise: if there is only one form field in the page, and it is in error, how do we check the rule?

Limiting the rule to a specific pair of inputs (assuming the previous concern is addressed) would, in my opinion, be too restrictive.

I think we should go with the second option. @Jym77, should we put this rule on hold till #1953 / #2045 are solved?

I think it's a list worth bringing to attention that this is another rule proposal that would benefit from being statefull, so that we are sure the solution can handle it.

WilcoFiers · 2024-08-29T16:58:01Z

Very out of date. Closing on behalf of @giacomo-petri

giacomo-petri added 2 commits September 5, 2022 10:14

Create error-field-is-not-identified-using-color-only

d5fd2ef

Update error-field-is-not-identified-using-color-only

4e177ad

giacomo-petri added the Rule Use this label for a new rule that does not exist already label Sep 5, 2022

giacomo-petri self-assigned this Sep 5, 2022

Jym77 self-requested a review September 6, 2022 09:15

Jym77 requested changes Sep 8, 2022

View reviewed changes

Jym77 requested a review from carlosapaduarte September 8, 2022 08:44

Jym77 mentioned this pull request Sep 8, 2022

"Error message describes invalid form field value" [36b590]: Possible improvements #1918

Closed

giacomo-petri added 15 commits September 8, 2022 13:34

Update error-field-is-not-identified-using-color-only

34e0526

Rename error-field-is-not-identified-using-color-only to error-field-…

d877bd9

…is-not-identified-using-color-only-i3z4v8.md

Update error-field-is-not-identified-using-color-only-i3z4v8.md

42cd956

Update error-field-is-not-identified-using-color-only-i3z4v8.md

fd8d6ec

Update error-field-is-not-identified-using-color-only-i3z4v8.md

27f4474

Update error-field-is-not-identified-using-color-only-i3z4v8.md

ed6ad31

Update error-field-is-not-identified-using-color-only-i3z4v8.md

84e5bc2

Update error-field-is-not-identified-using-color-only-i3z4v8.md

7e646ad

Update error-field-is-not-identified-using-color-only-i3z4v8.md

4c0fcf2

Update error-field-is-not-identified-using-color-only-i3z4v8.md

1125715

Update error-field-is-not-identified-using-color-only-i3z4v8.md

eb7163b

Update error-field-is-not-identified-using-color-only-i3z4v8.md

eae9d70

Update error-field-is-not-identified-using-color-only-i3z4v8.md

3a8e604

Update error-field-is-not-identified-using-color-only-i3z4v8.md

bac7869

Update error-field-is-not-identified-using-color-only-i3z4v8.md

72d095c

giacomo-petri requested a review from Jym77 September 26, 2022 10:23

Jym77 requested changes Oct 7, 2022

View reviewed changes

giacomo-petri and others added 4 commits October 7, 2022 10:20

Apply suggestions from code review

fef2a75

Co-authored-by: Jean-Yves Moyen <jym@siteimprove.com>

Update error-field-is-not-identified-using-color-only-i3z4v8.md

ae35c3e

Update error-field-is-not-identified-using-color-only-i3z4v8.md

9940265

Update _rules/error-field-is-not-identified-using-color-only-i3z4v8.md

a5198c9

Co-authored-by: Jean-Yves Moyen <jym@siteimprove.com>

giacomo-petri requested a review from Jym77 January 17, 2023 10:34

Jym77 previously requested changes Jan 19, 2023

View reviewed changes

giacomo-petri added the manual rule label Feb 7, 2023

giacomo-petri added 2 commits May 9, 2023 18:10

Update error-field-is-not-identified-using-color-only-i3z4v8.md

1998047

Merge branch 'act-rules:develop' into giacomo-petri-error-field-not-i…

68d9853

…dentified-using-color-only

Update error-field-is-not-identified-using-color-only-i3z4v8.md

1ef6b3d

giacomo-petri requested a review from Jym77 May 9, 2023 16:19

Jym77 requested changes May 25, 2023

View reviewed changes

WilcoFiers closed this Aug 29, 2024


		## Applicability

		This rule applies to any error field that is [visible][].


		## Applicability

		This rule applies to any [visible][] [input error][] together with its visual context.


		## Expectation

		Each test target, together with its visual context, identify the presence of errors not only through color differences.


		#### Passed Example 2

		The "phone number" `input` value doesn't match the `type` value. Its visual error indicator (the first couple of `label` and `input` elements) conveys the presence of an error through its CSS properties:


		#### Failed Example 1

		The "first name" `input` value is empty. Its visual error indicator (the red border color of the first `input` element) conveys the presence of an error through color (hue) differences only.


		## Applicability

		This rule applies to each [HTML element][] that is [visible][] and has one of the following [semantic roles][]:

	This rule applies to each [HTML element][] that is [visible][] and has one of the following [semantic roles][]:
	This rule applies to any [HTML element][] that is [visible][] and has one of the following [semantic roles][]:


		## Expectation

		Each test target either has no [visible][] [form field error indicators][], or at least one of the [visible][] [form field error indicators][] identifies the presence of errors not only through color differences.

Error field is not identified using color differences only #1914

Error field is not identified using color differences only #1914

Uh oh!

Conversation

giacomo-petri commented Sep 5, 2022

Pull Request Etiquette

When creating PR:

After creating PR:

When merging a PR:

How to Review And Approve

Uh oh!

Jym77 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

giacomo-petri Sep 8, 2022 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jym77 left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Jym77 left a comment

giacomo-petri Sep 8, 2022 •

edited

Loading

Jym77 May 25, 2023 •

edited

Loading

giacomo-petri May 25, 2023 •

edited

Loading