Safari (WebKit) Support

[amirrustam]

Current Status - last updated August 4, 2022

✅ Prioritized and on the roadmap
✅ Doing exploratory work to assess requirements and timelines
✅ Under active development
ℹ️ Cypress currently supports Firefox and Chrome-family browsers (including Edge and Electron).

Objective

Add support for running Cypress tests within WebKit, which is the engine used by the Safari web browser.

Task List

• Initial PR to add development-only support: chore: WebKit support (development-only) #15533
• Fix required driver/other functionality
  • Network functionality chore(webkit): fix WebKit network-related driver tests #23232
  • Keyboard tests (type*.cy.js) chore(webkit): driver updates for clicking/typing actions and related tests #23522
  • Input tests (click.cy.js, focus tests...) chore(webkit): driver updates for clicking/typing actions and related tests #23522
  • Stack traces chore(webkit): update error stack parsing and related system tests #23730
  • Check component testing
  • Video recording chore(webkit): add video recording #23579
  • Handle downloads chore(webkit): add before:browser:launch and download support #23662
  • Fix global mode detection of npm module feat: gate WebKit behind experimentalWebKitSupport in prod #23711
  • Fix open mode close chore(webkit): add before:browser:launch and download support #23662
  • Disable forceNetworkError feat: gate WebKit behind experimentalWebKitSupport in prod #23711
  • Fix cross-origin unhandledRejection events: chore(webkit): working around WebKit bug by patching Error #23588
• Possibly fix less critical functionality
  • "Network" GraphQL error fix experimentalWebKitSupport: Fix "[Network] undefined" error in dev #23809
  • Multi-domain implementation feat(webkit): fix multidomain driver tests in WebKit #23442
  • Fix websocket polling/proxying/allowedSockets experimentalWebKitSupport: Fix proxying of Cypress internal websocket #23807
  • Fix devicePixelRatio experimentalWebKitSupport: Set devicePixelRatio #23808
  • Fix/skip focus from app
• Documentation - limitations, Linux dependencies docs: document experimentalWebKitSupport cypress-documentation#4710
• Add experimental flag, make available in production, update public types feat: gate WebKit behind experimentalWebKitSupport in prod #23711

[foolip]

I've done a fair bit of work on getting Safari working well with web-platform-tests (a conformance test suite for browsers with a dashboard) and know a little bit about what kinds of problems you might run into.

I know that Cypress doesn't use WebDriver but rather CDP (at least for Chrome) but can someone sketch roughly what the requirements of the protocol are? I've had a look at CONTRIBUTING.md and browser support docs but couldn't find any high-level information about Cypress internals.

I ask because I'd like to understand if it would be at all feasible to communicate with Safari using safaridriver, or what the basic "API" or protocol between Cypress and a browser needs to be.

[kuceb]

hi @foolip
we would need the following from either extension/devtools APIs:

• get/set/clear cookies by domain
• screenshot page
• record video of page (this can be done via getUserMedia api if it supports capturing the browser tab, which we do in firefox)

in addition, we would also need:

• a way to set a proxy server for all network traffic
• a way to bypass the invalid ssl cert screen/prompt

everything else is done via JavaScript

[foolip]

Thanks @bkucera, that's a great list!

get/set/clear cookies by domain

It sounds like you need to get/set/clear cookies for domains other than the one currently loaded, is that right? It looks like the only way of doing that using WebDriver/safaridriver would be to load a page from that domain and then use the endpoints with an implicit domain, but that's not great.

record video of page

I don't think this is possible, but I did have fun recording the whole macOS VM in Azure Pipelines using screencapture once. One can also record a specific window, but I presume you want the content area without decorations.

a way to set a proxy server for all network traffic

I'm pretty sure this is not possible yet.

a way to bypass the invalid ssl cert screen/prompt

I wouldn't be surprised if this is already possible with some setting or command line argument, but I don't know what it is. Our setup for web-platform-tests is a bit special, we update /etc/hosts and add a certificate to the system certificate store. It's a bit much, and also requires that you know what the domains under test are before you start the tests, which is probably not the case for Cypress.

[kaushalyap]

@amirrustam We got Firefox and Edge support recently, love to see, Safari support soon since Safari have

• 17.21% of global browser market share (2nd most used browser, second only to Chrome) : source
• 36.28% USA browser market share (2nd most used browser in USA, second only to Chrome) : source
• 33.48% UK browser market share (2nd most used browser in UK, second only to Chrome) : source

So please support Safari too.

[kelly-tock]

Yes for a lot of responsive sites mobile safari is a must have and can be the majority of the users.

[clockelliptic]

i want so badly to use Cypress, but I simply must use WebDriver because Safari support is a requirement over here

[lughino]

I’d love to see a quick decision around this topic, especially now that playwright is gaining a lot of popularity.

Cypress team, please prioritise this compatibility.

[it-sha]

I would suggest to my company and all clients to use Cypress if Safari support is implemented

[lkoorsse-tails]

Starting using cypress instead of selenium, great advantages over selenium, however Safari browser is a major requirement, would love to see cypress support for this soon to continue using cypress

[AlexanderTsipin]

same, using Wdio just because safari support, management requirements.
Please implement even a basic safari runner.

[flenerob]

Please add support to Safari !!!

[jhuesos]

Could this issue to be locked for comments. Adding the "I really need this message" does not really help and it creates a new push notification that everyone subscribed receives and it is annoying. Do you want to reflect that you need this? Click 👍 on the issue.

[mydiemho]

In the meantime, could we update the error to show that Safari is not supported in Cypress rather than the current misleading error that Safari isn't installed on the system? I wasted a few hours trying to debug my CI because I thought Safari was not installed.

Here's the current error message

> npx cypress run --browser  "safari"

Can't run because you've entered an invalid browser name.

Browser: 'safari' was not found on your system.

Available browsers found are: chrome, firefox, electron

[estefafdez]

I would love to have that!

[AlexanderTsipin]

Found Solution: TestCafe supports Safari and iphone, real devices and emulators

[wojciech-sygier]

I'm gonna add +1 to people requiring Safari support in Cypress. We're currently experimenting with different frameworks and we narrowed down our selection to TestCafe and Cypress. While most of us want Cypress, we also need the ability to test on Safari (due to ~30% of our users using it, 77% of which are using mobile version) along with other (currently supported) browsers.

Can we have at least some rough estimate for when any work would start? SoonTM is too broad and we can't really risk rewriting our tests if we don't get it in, let's say, a year or so.

[0xc0d3r]

@wojciech-sygier We too have the similar user segment and we took a call to proceed with TestCafe and it is working for us. We are using BrowserStack to run the tests on real devices and it is super cool.

[jennifer-shehane]

We evaluate current feature work on a number of factors and have to make tough decisions on which to currently work on. This is not part of our current feature work. This is not to say that it will never be delivered. We will comment in this issue when that decision changes. We do not have a rough estimate on when this will be delivered.

We are always re-evaluating what to work on next and feedback from people using Cypress is a big part of that evaluation. We welcome comments in this thread on how this feature would impact your testing.

[jennifer-shehane]

@mydiemho This error message was updated to be clearer about which browsers are supported in Cypress 4.8.0. #7375

[kelly-tock]

Where can we see the current feature work that is prioritized?

[kopepasah]

First off, thanks for the Cypress team for finally jumping on this, as they can see it IS prudent to the future success of this project to support testing in Safari.

After reading through this thread, I personally get the feeling that the Cypress team was very reluctant to implement this feature, even going as far to think we want this feature to appease C-suite executive requests. However, this cannot be further from the truth, as Safari usage has been growing consistently year over year, and is now the second most used browser on the web.

We are asking for this feature because we need to run tests in browsers our users use, not our developers. While the challenges of testing in Safari (and Webkit) are present, the thought of not testing in these scenarios because the tooling is hard is a no-go for many companies (there's a reason Microsoft uses Playwright.dev).

As I stated before, I am excited to see support for Safari coming, but I do hope the Cypress team learns from this experience and prioritizes testing where users are, instead of what tools developers use.

[weyert]

After reading through this thread, I personally get the feeling that the Cypress team was very reluctant to implement this feature, even going as far to think we want this feature to appease C-suite executive requests. However, this cannot be further from the truth, as Safari usage has been growing consistently year over year, and is now the second most used browser on the web.

As I stated before, I am excited to see support for Safari coming, but I do hope the Cypress team learns from this experience and prioritizes testing where users are, instead of what tools developers use.

My experience is that adding WebKit support to Cypress wasn’t straightforward at all. Of course, I never knew the code base as well as Cypress employees but this browser behaves in some ways differently from Firefox and Chrome in my experience so you end up with special logic for WebKit. For example, something as the type() action in Cypress. So I can understand reluctance, but I am glad it’s coming.

[romankhomitskyi]

After reading through this thread, I personally get the feeling that the Cypress team was very reluctant to implement this feature, even going as far to think we want this feature to appease C-suite executive requests. However, this cannot be further from the truth, as Safari usage has been growing consistently year over year, and is now the second most used browser on the web.

As I stated before, I am excited to see support for Safari coming, but I do hope the Cypress team learns from this experience and prioritizes testing where users are, instead of what tools developers use.

My experience is that adding WebKit support to Cypress wasn’t straightforward at all. Of course, I never knew the code base as well as Cypress employees but this browser behaves in some ways differently from Firefox and Chrome in my experience so you end up with special logic for WebKit. For example, something as the type() action in Cypress. So I can understand reluctance, but I am glad it’s coming.

I guess iframes in webkit it's a long shot

[weyert]

@flotwig @tbiethman
If I remember correctly, you somewhere needed to get the process id, to properly integrate the webkit browser in the Cypress system. My own code of Cypress suggests that the following should help getting access to that:

import { webkit } from 'playwright-webkit'

const browserServer = await webkit.launchServer({
    headless: false,
})

browserServer.addListener('close', () => {
    console.log(`Browser has been closed`)
})

const websocketEndpoint = browserServer.wsEndpoint()

console.log(`websocketEndpoint:`, websocketEndpoint)
console.log(`Process ID:`, browserServer.process().pid)

const browserInstance = await webkit.connect(websocketEndpoint, {})
const page = await browserInstance.newPage({ baseURL: 'https://tapico.io' })
page.goto('/')

If I run this the following console output is returned:

❯ node experiment.mjs
websocketEndpoint: ws://127.0.0.1:57402/c7675392af118656f013c2464b5a9564
Process ID: 94930

Now if you check the the processes it getting returned as expected:

❯ ps ax | grep 94930
94930   ??  Ss     0:00.00 /bin/bash /Users/weyert/Library/Caches/ms-playwright/webkit-1699/pw_run.sh --inspector-pipe --no-startup-window

In my version I never really cleanly got the video recording integrated, what I ended up, due to my lack of deep knowledge of the code base, is always record videos when video recording is enabled, and then delete it when the test succeeded. Never was able to get access to the raw frames that the code appeared to be expecting.

This approach probably won't work if you are running multiple test cases; you might need to chop up the video in between or something. Forgot the details about that.

I hope this helps, I will see if I make some PRs of working bits that I have that you still need to implement after I met my work deadline (e.g. downloads)

[flotwig]

If I remember correctly, you somewhere needed to get the process id, to properly integrate the webkit browser in the Cypress system. My own code of Cypress suggests that the following should help getting access to that:

@weyert Oh yeah, this is way better than how I was thinking of doing it, thanks. I was going to pull the full Cy process tree and try to identify the PW process using the command string. We need the pid for process_profiler and the "Focus" button in the launchpad. But there's a 3rd use for it. I noticed that await browser.close() can take several seconds to complete, so we might want to use process.kill(webkitPid) to end the WK process instead. So this is triply useful.

In my version I never really cleanly got the video recording integrated

Currently working on this, and it's taking somewhat of a refactor of how Cypress records video. Cypress expects to be able to start video recording anytime and to receive raw frame data, and PW does neither of those 😛

I will see if I make some PRs of working bits that I have that you still need to implement after I met my work deadline (e.g. downloads)

Great, if you do have time, the task list on this issue should be current. Feel free to ping me on Discord if you'd like to coordinate efforts.

[monkpit]

Closed means it's going to ship soon? 👀 we have had our eye on this issue in order to expand our Cypress tests to encompass Responsive pages on Safari. I'd be overjoyed if this is going to ship :) Thanks for all of your work!

[weyert]

Great job on closing this ticket. Looking forward seeing it announced :)

[richard-owusu]

Good job! When can we expect the announcement?

[flotwig]

Hey folks, this is coming out as experimental in today's release, 10.8.0. I'll comment back here once it's out.

[cypress-bot]

Released in 10.8.0.

This comment thread has been locked. If you are still experiencing this issue after upgrading to
Cypress v10.8.0, please open a new issue.

[flotwig]

Cypress 10.8.0 is now available with experimentalWebKitSupport!

See the docs for instructions on installing and enabling WebKit: https://docs.cypress.io/guides/guides/launching-browsers#WebKit-Experimental

There are some known issues under the experiment: webkit tag, please consult those before opening a new issue.

https://www.cypress.io/blog/2022/09/13/cypress-10-8-experimental-run-tests-in-webkit/