[css-images-3] change which images image-orientation applies to

[faceless2]

In #4165 it was resolved that EXIF orientation applies to all images by default. There was some discussion about which images the image-orientation property and talk of a resolution, but no actual resolution.

This matters because we've got two sets of WPT tests added in the last six months which disagree - see web-platform-tests/wpt#18549 (comment) - and different results in Firefox and Chrome (see a, b). So we need a once-and-for-all resolution on this.

@heycam made the same observation in #4165 (comment), and we'd like to implement this but can't until it's clarified.

So the question is: does the image-orientation property apply to "decorative" images, such as background-image and border-image? And if it does not (as currently specified), the second question is can we tightly define "decorative" images.

@zcorpan made a full list at web-platform-tests/wpt#18549 (comment)

CSS

• ::before/::after + content
• background-image
• list-style-image
• border-image
• cursor

SVG

• image

HTML

• img
• canvas drawImage() + getImageData()
• video poster
• input type=image
• embed
• object

load an image in an iframe
load an image in a top-level doc (this already respects EXIF in all browsers I believe)
Favicon link rel=icon / favicon.ico (manual tests?)
web app manifest icon (manual test?)

• I would expect "content" images to be those displayed from content, list-style-image (which generates content), SVG image, HTMLimg, embed, object, and video poster. The image-orientation property should apply to all of these.
• I would expect "decorative" images to be background-image, border-image and mask-image.
• image-orientation does not apply to cursor, top-level docs, favicon or web-app manifests.
• I'm slightly unsure about exactly where iframe fits.

cc @schenney-chromium

[heycam]

So the question is: does the image-orientation property apply to "decorative" images, such as background-image and border-image? And if it does not (as currently specified), the second question is can we tightly define "decorative" images.

Agree that that's what we need to answer.

The goal of the image-orientation property is to make it easy to turn off the new automatic re-orientation behavior, in case authors have content that breaks with the new behavior. (We had a few bug reports in Firefox of content that broke, due to images being used that had incorrect orientation metadata in them, probably because of tools that rotated the image data but left the metadata untouched. So it is useful for quickly fixing this. But these were all <img> elements.)

If image-orientation doesn't apply to decorative images, then there is no easy way to opt out of any breakage that's occurred. So I would prefer to have it apply to all content and decorative images that the element has, and then not bother with defining decorative images.

I don't have a strong opinion on cursor and resources linked to with <link> / <meta> elements, but I think it would be fine not to them apply there. It's unlikely people are using JPEGs with orientation data in them.

My view of orientation metadata is that it's an implementation detail of the image, and the only reason that we are allowing control of whether it is honored is to help authors manage this change in behavior.

[faceless2]

I agree with Cam on all his points. If EXIF applies to all images, then I think image-orientation has to as well.

[css-meeting-bot]

The CSS Working Group just discussed [css-images-3] clarify which images image-orientation applies to, and agreed to the following:

• RESOLVED: Make from-image's none value apply to background and any other images

The full IRC log of that discussion

<dael> Topic: [css-images-3] clarify which images image-orientation applies to
<dael> github: https://github.com//issues/5245
<dael> heycam: We have image orientation property that allows us turn off auto-re-orientation.
<dael> heycam: Some discussion about widening to all images a few months ago. Resolution from there was orientation meta data should be honored for decorative images. Makes sense.
<dael> heycam: Wasn't discussed if properties should be extended. As MIke noticed we have contradictory WPTs. Should image-orientation apply to decorative images?
<dino> q+
<dael> heycam: My feeling is it should b/c real purpose of property is let authors opt-out and it doesn't make sense to limit that to a subset of images
<Rossen_> ack dino
<dael> dino: Question is what would you do if want decorative to do one thing and non-decorative do another
<dael> heycam: You would need to introduce some other way of override orientation, maybe with image value syntax, or correct images
<dael> dino: Sort of leading question. In many cases will have to fix image or change content. Is it worth having this apply everywhere? I don't know
<dael> heycam: From experience of bug reports they were all content images. jpegs with image data rotated but tool didn't update meta data so new image-orientation made them wrong. Didn't get cases on decorative
<dael> fantasai: We had this discussion before and I believe resolution was only to apply to content. Idea was if we needed to apply to other we would introduce image notation syntax. This discussion was before auto xif but that was original discussion and conclusion
<dael> Rossen_: Is that 4165?
<dael> fantasai: older
<dael> fantasai: Let me change DoC
<smfr> s/change/see if it’s in the/
<dael> s/xif/EXIF
<fantasai> https://lists.w3.org/Archives/Public/www-style/2012Mar/0412.html
<dael> Rossen_: Where does this leave the current issue?
<dael> fantasai: Here's the issue. #50 in DoC. Conclusion was repalced elemtns and generated content but not background image
<dael> fantasai: Or any other images in CSS
<dael> fantasai: Images which are considered part of content are effected but no other
<dael> faceless2_: If purpose is undo exif makes sense to apply, but it's not weird to not. As long as it's documented.
<fantasai> "It applies only to content images (e.g. replaced elements and generated content), not decorative images (such as background-image)."
<dael> fantasai: Spec says applies to content not decorative.
<fantasai> https://drafts.csswg.org/css-images-3/#the-image-orientation
<dael> fantasai: Anything that's a replaced element in css model it applies to that. Everything else is not effected.
<chrishtr> q+
<Rossen_> ack fantasai
<Zakim> fantasai, you wanted to discuss history
<dael> heycam: Seems like notion of this property has changed over time. Original I want to set an orientation on an image. Now proprty is forI I have a problem caused by change in browsers and this gets me the old way back so I don't have to fix images I'm serving. In that PoV makes sense to apply as broadly as possible. Let's author set one property which goes to all images
<fantasai> This was issue 50 in the 2012 LC Disposition of Comments https://drafts.csswg.org/css-images-3/issues-lc-2012
<dael> chrishtr: I agree with heycam reasoning and that's what Chrome impl. There were cases where it was useful behavior for site compat
<smfr> webkit agrees with cameron too
<dael> fantasai: If we make this change it should only affect none value and others shouldn't apply to decorative
<Rossen_> ack chrishtr
<dael> heycam: I think we have resolution to remove other values.
<dael> fantasai: It's deprecated but has been impl and shipped in multiple impl, jsut nto browsers. We can remove from next spec level, but there needs to be a spec with those.
<dael> chrishtr: Who impl?
<dael> fantasai: Some printer based technology sihpped with onboard layout in printer chipset.
<dael> chrishtr: Okay so we say rotate ones won't apply to style images
<dael> fantasai: And support for that is marked as optional so browsers don't need to impl. But there needs to be spec for it
<dael> Rossen_: Any changes to L3 images?
<dael> fantasai: Yes if we want to make from-image and none value apply to background and other images we need to change spec to say it's all images associated with element
<dael> Rossen_: Is this going to be still valid for printer or are they okay b/c L2?
<dael> fantasai: They don't impl from-image I believe
<dael> Rossen_: Sorry, confused by statement that there were impl
<dael> fantasai: If we make change for other values than yes. THat's why I'm saying it shouldn't apply to the other values
<dael> Rossen_: Would that solution work for everybody
<dael> heycam: sgtm
<dael> Rossen_: Prop: Make from-image and none value apply to background and any other images?
<dael> fantasai: angle only applies to content
<dael> Rossen_: none value on from-image
<dael> Rossen_: Additional comments?
<dael> smfr: Can we resolve on cursor behavior and linked meta that are in GH issue?
<dael> Rossen_: Would this be the exception on cursor?
<dael> smfr: Makes sense for cursor to have same. Link and meta I don't know since usually don't apply CSS to them
<dael> fantasai: Not sure what a link or meta element would do
<dael> heycam: Some cases about favicons and similar images
<dael> smfr: First comment in issues has list of interesting items like embed object. We need to be explicit which of these we include
<dael> chrishtr: Canvas is an imparative API
<dael> smfr: Not sure abotu canvas I think right is API to expose exif to JS or extend draw image API
<tantek> presumably you can put images on link or meta via background-image or ::before and content property etc.
<dael> fantasai: Rest it should apply, every image associated to the element is effected as far as CSS is concerned. If it's a replaced element it's applied. Decorative elements would include background iamge and cursor
<dael> smfr: Source graphic in SVG
<dael> fantasai: Replaced element, isn't it?
<dael> smfr: I think it's paint source or whatveer it's called
<dael> heycam: I think that's rendered content of some subelement on dom tree
<dael> fantasai: Do you do image orientation of source, using, or neither element
<dael> heycam: Similar to canvas-draw-image b/c can reference another image element. Have minor incompat on where to look already. I think those cases can be handled in specs that define what it's referencing
<dael> smfr: Agree. We can resolve on what we discussed and say SVG may need refining
<dael> heycam: I'm happy to make cursor effected
<dael> Rossen_: Should we try to resolve on this and heycam or someone can open an issue on SVG to makes sure there's no additional items to associate to this decision?
<dael> smfr: sgtm
<smfr> s/smfr/heycam/
<dael> Rossen_: Still previous resolution. Objections?
<dael> RESOLVED: Make from-image's none value apply to background and any other images

[fantasai]

Because the spec was pretty clear that it does not apply to background images etc., only to replaced elements (including generated content).