Special-casing file upload controls in keyboard-interactability check

[andreastt]

It is apparently the case that a range of “modern” JS frameworks hides <input type=file> and provides indirection for selecting files via alternate UX elements. When using Element Send Keys to upload files to the file upload element WebDriver will typically complain that the element is not interactable, because the JS framework will have hidden the element from view (display: none).

I have not yet received a reproducible test case for this, so it’s unclear to me if it is possible to write automation code to interact witht he UX element to indirectly set the file on the file upload element or not. If it is true that this is impossible, we should consider how we can make it possible to use Element Send Keys to upload files when the file upload element is hidden from view.

At the moment I can’t think of any better solutions that to special case <input type=file> in the keyboard-interactability check so that we always treat it as interactable. This would enable users to upload files to non-displayed elements.

Can someone who is more JS-savvy confirm how file upload control indirection works in practice?

See also:

• For input[type=file], should allow sendKeys even when display=none mozilla/geckodriver#1173
• Element <input type="file"> is not reachable by keyboard mozilla/geckodriver#1184

[shs96c]

I think it's wise to get a reproducible test case for this, and then see if we can interact with the alternative elements. The two cases listed there show people attempting to interact with the file input element, but not (I think) with whatever the user actually interacts with to upload elements.

I wonder what chromedriver does, since some folks are reporting that works for them.

[twalpole]

A common approach is to make the file input transparent and overlay it over the frameworks alternative elements. That way a click "on" the alternative elements actually clicks the file input and opens the file chooser dialog. If the file chooser dialog was treated similar to a JS prompt dialog and allowed to set the files being selected when it displays then it would solve all the issues (would require WebDriver spec update though). The issue with just treating a hidden file input as always accessible (which chromedriver appears to do) is that you're not then actually testing the page as a user would be able to use the page since it may be able to add files when a user really should not be able to.

[shs96c]

I like the idea of being able to interact with the file upload dialog as if it were a modal prompt. I do worry about how easy it is to implement.

[kerrykimbrough]

As the one who initiated this issue at mozilla/geckodriver, I agree that the problem would be better solved by a standard WebDriver API for recognizing a file chooser dialog and setting the file it returns, as if entered by a user.

I don't think a "reproducible case" will reveal any better solutions. As twalpole suggests, whenever the <input type=file> is hidden, whatever users actually interact with must ultimately act directly on the DOM to activate that element and present the file chooser dialog. But at that point, a WebDriver client program is helpless.

The sendKeys hack is a way avoid the file chooser dialog altogether. But it would be better if a WebDriver client could control it.

[twalpole]

@kerrykimbrough Just a quick fix - "setting the FILES it returns" -- anything around the file input needs to support the selection of multiple files to be useful for modern testing.

[AlexanderArendar]

@shs96c I am not sure what exactly chromedriver does technically, but for me as a developer of automated tests, it gives ability to automate. My concrete example right now is a project where users can create events, part of events data is event logo, event favicon and event background image. So there are just 3 file inputs with all the Agular JS/CSS stuff wrapped around them which makes the inputs themselves not intractable. With chromedriver I just can use sendKeys and have my basic test case of event creation/editing automated. With geckodriver I just check if(driver.instanceOf[FirefoxDriver]) and fail major part of test with the message "can not be automated in Firefox".
So you must understand that such cases are typical and you don't need to philosophize if it would be better to have a reproducible case or if we automation engineers are really trying to automate exactly what users are doing or not. With the current approach geckodriver JUST DON'T LET YOU AUTOMATE TYPICAL CASES FOR FILE UPLOAD. We are not living in 1990-s where just pure HTML file inputs are common thing.
Don't get me wrong, I am not complaining. If you look I was actively commenting the source tickets referenced in this one. But already a month passed and the ticket is still in a state where nothing helpful was suggested. So if you guys take path to discuss for months I believe me and others will just consider geckodriver not suitable and that would be a real PITA because no alternatives exist for Firefox. After all geckodriver should be focused on providing way to automate, not to restrict basic use cases of automation.

[twalpole]

@AlexanderArendar Yes - chromedriver makes it work, but you need to realize that you're not actually testing the page then. If all the Angular JS/CSS stuff wrapped around the inputs was broken, hidden, etc your tests would still pass even though a user of the page would be unable to use it - not exactly the best position to be in.

[AlexanderArendar]

@twalpole, I agree with this. But again, in 90% or more while things are not broken I would be able to test if the user can create/edit event, if user can create event request, etc. At the moment I just can't do this in Firefox at all. Even when nothing ever breaks in input file UI wrappers I am blocked and can't cover really big part of my app functionality.
In my other project there was similar situation with CSV upload for price management module. I've been working on this project for a while and again, nothing ever broken in particular UI controls for CSV upload, but I am not able to cover tests for prices import/update in Firefox.
So what good is that for to defend such approach?

[twalpole]

@AlexanderArendar "what good is that for to defend such approach?" - the flip side to that is what good is it to argue for an approach that means tests don't let you know when your app is broken? The reality is there needs to be a better approach than just special casing file inputs to ignore visibility.

Additionally this repo is for the W3C WebDriver spec (not geckodriver). Expecting any W3C spec - which relies on consensus of multiple people - to be changed in a month is wishful thinking at best. If you need to test your pages using geckodriver today then in your tests you'll need to modify the CSS of the file inputs using JS so they become visible.

[AlexanderArendar]

@twalpole, in real world, it is very often that you cannot just "modify the CSS of the file inputs" because you work with already existing app and how UI looks is not something QA can and should impact.
Also I am not against the "better approach". But question is when will it materialize, right? I am just a man from the field who tries to use existing toolkit and provides some feedback.
And finally, recommending me to "modify the CSS of the file inputs using JS" would not it then make me test not what real users have in their browsers? Looks like your argument is then biased.
Probably I have nothing more to add, so have a good day and I hope W3C team will eventually come up to some good solution.

[twalpole]

@AlexanderArendar Of course you can ALWAYS modify the CSS of the file input via JS, as I mentioned, from inside your test - It's even mentioned in the first geckodriver issue linked in this issue description. Yes this would make it test what real users can't do, but that's exactly what the current chromedriver approach you seem to want put into the spec does too, but in this case you would have to proactively do something thereby indicating you know you're not actually testing what a user can do.

[andreastt]

This discussion appears to have derailed quite substantially. Let me try to bring it back on track.

@shs96c said he liked the idea of being able to interact with the file upload dialogue “as if it were a modal prompt”. I can say straight away that this is not viable because browsers use native widget toolkits to present the dialogue over which the browser has no control. Control is only returned to the browser event loop after the dialogue closes.

I think if we were designing this API from scratch a new endpoint such as /session/{session id}/element/{element id}/upload would make sense. This would bypass the keyboard interactability checks associated with the Element Send Keys command, and also avoid the cognitive overload that interacting with <input type=text> has a different meaning and effect than interacting with <input type=file>. I think many users are also confused by how multiple repeated calls to <input type=file multiple> appends to the FileList.

However, we need to deal with realities. It is expected behaviour that a conforming WebDriver implementation must be able to upload a file by using Element Send Keys to an <input type=file> element. It is incorrect, as some of the comments in this discussion imply, that geckodriver does not support this. The current situation is that chromedriver does not conform to the specification by honouring the keyboard-interactability promise of Element Send Keys, but the discussion is about whether we should change the specification to match chromedriver’s behaviour.

In order to move forward on this, we require two things:

1. Clarity into what chromedriver does when Element Send Keys is called on <input type=file> and the form control is not keyboard-interactable, e.g. display: none.
2. A reproducible test case showcasing the Angular/React problem.

Perhaps @kereliuk can help explain how chromedriver works, and it would also be useful with input from the other spec editor, @AutomatedTester.

[QAdamek]

@andreastt Good day.

My name is Adam Domzalski and currently I work as a QA / automation guy.
Since yesterday I have EXACTLY the same problem:

• Frontend written in React.JS
• <input type="file" multiple="" style="display: none;">
• Chrome can upload file using sendKeys()
• Firefox can't

If you want me to get some data I would love to help.