Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(allure-cypress): new real-time lifecycle, hooks, and test plan #1109

Merged
merged 10 commits into from
Aug 9, 2024

Conversation

delatrie
Copy link
Collaborator

@delatrie delatrie commented Aug 9, 2024

Context

The PR contains the following major changes of Allure Cypress:

Real-time reporting

Previously, Allure Cypress reported the results in a batched fashion when the test run ended. The new implementation employs:

  • an updated set of messages
  • message queueing on the browser side
  • queue flushing to the Node side in the plugin's afterEach hook
  • the tail flushing in the plugin's after hook

The messages are flushed using Cypress'es tasks mechanism. It works reliably only in hooks, that's why index.ts defines afterEach and after hooks.

In interactive mode, the spec reporting is finalized in Mocha's run end event. In cypress run mode the finalization is done in Cypress'es after:spec event.

environment.properties and categories.json are reported in after:run.

New hooks implementation

The new implementation supports all types of hooks. Spec-level after hooks are supported by intercepting the after calls and injecting the plugin's after hooks right next to user-defined ones. That allows us to issue the reporting chronologically after the user-defined after is completed and its corresponding messages are queued.

New test plan implementation

The previous implementation requested the test plan from the browser side through the Cypress task. It applies the test plan in a beforeEach hook and skipped tests with this.skip().

The downside is that, while correctly skipping the tests, this approach fails to disable the associated hooks, meaning all the before and after hooks will be executed. Given those hooks often contain tasks like spinning up test servers, initializing databases, etc, that may slow down the execution substantially.

The new implementation uses Cypress environment variables to pass the test plan down the specs. That allows us to get it early enough to apply before the spec-level suite runs, disabling both the tests and their hooks (even the ones defined on the spec-level).

The only requirement for a user is to pass the Cypress configuration to allureCypress:

import { defineConfig } from "cypress";
import { allureCypress } from "allure-cypress/reporter";

export default defineConfig({
  e2e: {
    setupNodeEvents: (on, config) => {
      allureCypress(on, config, { /* Allure options go here */ });
      return config;
    },
    // ...
  },
});

Note

For backward compatibility, passing Allure configuration as the 2nd argument also works.

fixes #1078
fixes #1079

Other changes

  • New README for Allure Cypress

Checklist

  - fix spec-level after hook test
  - add some missing assertions
  - fix some test names
  - add testplan test and hook exclusion checks
  - remove reporter instance test (the use case is meaningless)
…tion

The notable differences compared to the previous batched reporting:
  - results are reported in real-time
  - all types of hooks are supported in all kinds of scopes
  - fixture scopes are fixed (some more tests are needed)
  - a test plan also disables hooks

Fixes #1078 and #1079
Also:
  - add more conveniently named event handlers for Node event forwarding
  - add docstrings for event handllers
@delatrie delatrie added the type:new feature New feature or request label Aug 9, 2024
@delatrie delatrie marked this pull request as ready for review August 9, 2024 12:07
* plugins. More info [here](https://github.com/allure-framework/allure-js/blob/main/packages/allure-cypress/README.md#setupnodeevents-limitations).
* @param spec The first argument of the `after:spec` event.
* @param results The second argument of the `after:spec` event.
* @example
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Better to wrap up @example content into markdown backticks ```

* });
*/
// eslint-disable-next-line @typescript-eslint/no-unused-vars
onAfterRun = (results: CypressCommandLine.CypressFailedRunResult | CypressCommandLine.CypressRunResult) => {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's remove unused variables with the eslint comments. If we don't use them now, we won't use them in future

@baev baev merged commit 4fb78e9 into main Aug 9, 2024
10 checks passed
@baev baev deleted the cypress-lifecycle-rework branch August 9, 2024 18:15
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type:new feature New feature or request
Projects
None yet
3 participants