0.1110.0 has breaking changes with no migration guide?

[jsjoeio]

We adopted playwright-test at the suggestion of @dgozman on April 8. It's been awesome and we're really happy we switched over from jest-playwright ♥️

However, now we see that this project is not considered ready for production as of 22 days ago.

And now that playwright v1.11.0 is out, we were planning to upgrade but it seems there are breaking changes?

As a user, it's hard to know if I should file this issue here, or in playwright or folio.

Before, we had a configuration file like this:

config.js

Click to toggle contents of `config.js`

import {
  ChromiumEnv,
  FirefoxEnv,
  WebKitEnv,
  test,
  setConfig,
  PlaywrightOptions,
  Config,
  globalSetup,
} from "@playwright/test"
import * as crypto from "crypto"
import path from "path"
import { PASSWORD } from "./utils/constants"
import * as wtfnode from "./utils/wtfnode"

// Playwright doesn't like that ../src/node/util has an enum in it
// so I had to copy hash in separately
const hash = (str: string): string => {
  return crypto.createHash("sha256").update(str).digest("hex")
}

const cookieToStore = {
  sameSite: "Lax" as const,
  name: "key",
  value: hash(PASSWORD),
  domain: "localhost",
  path: "/",
  expires: -1,
  httpOnly: false,
  secure: false,
}

globalSetup(async () => {
  console.log("\n🚨 Running globalSetup for playwright end-to-end tests")
  console.log("👋 Please hang tight...")

  if (process.env.WTF_NODE) {
    wtfnode.setup()
  }

  const storage = {
    cookies: [cookieToStore],
  }

  // Save storage state and store as an env variable
  // More info: https://playwright.dev/docs/auth?_highlight=authe#reuse-authentication-state
  process.env.STORAGE = JSON.stringify(storage)
  console.log("✅ globalSetup is now complete.")
})

const config: Config = {
  testDir: path.join(__dirname, "e2e"), // Search for tests in this directory.
  timeout: 60000, // Each test is given 60 seconds.
  retries: 3, // Retry failing tests 2 times
  workers: 1,
}

if (process.env.CI) {
  // In CI, retry failing tests 2 times
  // in the event of flakiness
  config.retries = 2
}

setConfig(config)

const options: PlaywrightOptions = {
  headless: true, // Run tests in headless browsers.
  video: "on",
}

// Run tests in three browsers.
test.runWith(new ChromiumEnv(options), { tag: "chromium" })
test.runWith(new FirefoxEnv(options), { tag: "firefox" })
test.runWith(new WebKitEnv(options), { tag: "webkit" })

See file here.

Breaking Changes

Here are the breaking changes we've noticed:

• folio prints error --config=config.ts https://github.com/microsoft/folio/issues/210#issuecomment-841415055
• @playwright/test no longer exports ChromiumEnv, FirefoxEnv, WebKitEnv, setConfig, PlaywrightOptions, Config, globalSetup

We completely understand that this is an early project and we're happy to support and use it because it makes end-to-end testing easier for us, but would love if you could help us by documenting any breaking changes and providing a proper guide to migrate/upgrade appropriately.

[dgozman]

@jsjoeio Sorry for confusion! You have migrated to the experimental version, and that gave us a lot of feedback - thank you! We are busy addressing that feedback and more, and meanwhile released 0.1110.0 to keep up with the recent Playwright release (using the old api and old underlying folio framework).

Configs and new stuff will be available at the next big release for playwright-test, so you won't have to update much, since you are already on the experimental version. Currently, we are developing under next label on the npm to avoid frequent breakages. With that release we'll also publish migration guide.

Hopefully, this clarifies the situation. I am happy to answer any questions here.

[jsjoeio]

Thanks for the quick reply. Totally make sense.

With that release we'll also publish migration guide.

Really glad to hear that.

next big release for playwright-test

I'll subscribe to the Releases so I get notified when that comes out. Thanks @dgozman!