Added ./tests/qa dir with actual tests for v7.9.0

- 431 actual tests (Playwright, typescript)
- Reporting to Testrail
- See README.md for more info

tests/qa/.env.example

@@ -0,0 +1,24 @@
+# MUtkin test singlesite, PHP 8.1
+# playwright-utils config
+WP_BASE_URL='https://'
+WP_USERNAME=
+WP_PASSWORD=
+WP_BASIC_AUTH_USER=
+WP_BASIC_AUTH_PASS=
+STORAGE_STATE_PATH='./storage-states'
+STORAGE_STATE_PATH_ADMIN='./storage-states/admin.json'
+
+# WooCommerce API config
+WC_API_KEY=
+WC_API_SECRET=
+
+# Mollie
+MOLLIE_LIVE_API_KEY=
+MOLLIE_TEST_API_KEY=
+
+# Testrail
+TESTRAIL_URL=
+TESTRAIL_USERNAME=
+TESTRAIL_PASSWORD=
+TESTRAIL_PLAN_ID=
+TESTRAIL_RUN_ID=

tests/qa/.gitignore

@@ -0,0 +1,13 @@
+.idea
+.vscode
+node_modules
+playwright-utils
+playwright-report
+playwright/.cache
+storage-states
+test-results
+package-lock.json
+playwright-wordpress-credentials.json
+
+.env*
+!.env.example

tests/qa/.npmrc

@@ -0,0 +1 @@
+@inpsyde:registry=https://npm.pkg.github.com

tests/qa/README.md

@@ -0,0 +1,216 @@
+# Mollie Tests Repo
+
+Mollie Playwright tests. Depends on [`@inpsyde/playwright-utils`](https://github.com/inpsyde/playwright-utils) package.
+
+__\* - Currently the `@inpsyde/playwright-utils` needs to be installed locally (see section _Installation for local development_ below) and switched to branch `work/misha`.__
+
+## Repo structure
+
+- `resources` - files with test-data, images, project related installation packages, types, etc.
+
+- `tests` - test specifications. For payment plugins contains following folders:
+
+	- `01-plugin-foundation` - general tests for plugin installation, uninstallation, activation, deactivation, display of plugin in __WooCommerce -> Settings -> Payments__.
+
+	- `02-merchant-setup` - tests for connection of current plugin instance to the payment system provider API via merchant (seller) credentials.
+
+	- `03-plugin-settings` - tests for various plugin settings, may include assertions of settings effect on frontend.
+
+	- `04-frontend-ui` - tests for plugin UI on frontend: display of payment buttons, display of payment methods depending on customer's country, etc.
+
+	- `05-transaction` - tests of payment process. Typically include: adding products to cart as precondition, payment (transaction) process, assertions on order received page, dashboard order edit page, payment via payment system provider API.
+
+	- `06-refund` - tests for refund transactions. Typically include: finished transaction as precondition, refund via payment system provider API on dashboard order edit page, assertion of refund statuses.
+
+	- `07-vaulting` - tests for transactions with enabled vaulting (saved payment methods for registered customers). Ability to remember payment methods and use them for transactions.
+
+	- `08-subscriptions` - tests for transactions for subscription products. Requires WooCommerce Subscriptions plugin. Usually available to registered customers and also includes vaulting and renewal of subscription (with automatic payment). WooCommerce Subscriptions plugin (can be [downloaded here](https://woocommerce.com/my-account/downloads/), login credentials in 1Password).
+
+	- `09-compatibility` - tests for compatibility with other themes, plugins, etc.
+
+	\* - folders are numerated on purpose, to force correct sequence of tests - from basic to advanced. Although each test should be independent and work separately, it is better to start testing from `plugin-foundation` and move to more complex tests.
+
+	\*\* - folders and numeration can be different, based on project requirements.
+
+- `utils` - project related utility files, built on top of `@inpsyde/playwright-utils`.
+
+	- `admin` - functionality for operating dashboard pages.
+
+	- `frontend` - functionality for operating frontend pages, hosted checkout pages (payment system provider's pages).
+
+	- `test.ts` - declarations of project related test fixtures.
+
+	- other project related functionality, like helpers, APIs, urls.
+
+- `.env`, `playwright.config.ts`, `package.json` - see below.
+
+## Installation of `@inpsyde/playwright-utils`
+
+### Installation as a node package
+
+1. Remove `"workspaces": [ "playwright-utils" ]` from `package.json`.
+
+2. In the root of the monorepo run following command:
+
+```bash
+npm run setup:tests
+```
+
+### Installation for local development
+
+1. Add `"workspaces": [ "playwright-utils" ]` to `package.json`.
+
+2. Delete `@inpsyde/playwright-utils` from `/node_modules`.
+
+3. In the root of the monorepo run following command:
+
+	```bash
+	git clone https://github.com/inpsyde/playwright-utils.git
+	```
+
+	[`@inpsyde/playwright-utils`](https://github.com/inpsyde/playwright-utils) repository should be cloned as `playwright-utils` right inside the root directory of monorepo.
+
+4. Restart VSCode editor. This will create `playwright-utils` instance in the source control tab of VSCode editor.
+
+5. Run following command:
+
+	```bash
+	npm run setup:utils
+	```
+
+6. `@inpsyde/playwright-utils` should reappear in node_modules. Following message (coming from `tsc-watch`) should be displayed in the terminal:
+
+	```bash
+	10:00:00 - Found 0 errors. Watching for file changes.
+	```
+
+7. If you plan to make changes in `playwright-utils` keep current terminal window opened and create another instance of terminal.
+
+## Project configuration
+
+Project from the monorepo requires a working WordPress website with WooCommmerce, `.env` file and configured Playwright.
+
+1. [SSE setup](https://inpsyde.atlassian.net/wiki/spaces/AT/pages/3175907370/Self+Service+WordPress+Environment) - will be deprecated in Q1 of 2025.
+
+2. Tested user with Administrator role should be created
+
+2. In the Dashboard navigate to __Settings -> Permalinks__ and select `Post name` in __Permalink structure__ for correct format of REST path.
+
+3. Install __Storefront__ theme.
+
+4. Install __WooCommerce__ plugin.
+
+5. In __WooCommerce -> Settings -> Advanced -> REST API__ create _Consumer Key_ and _Secret_ with Read/Write permissions and store them in `.env`.
+
+6. To avoid conflicts make sure any other payment plugins are deleted.
+
+7. Configure `.env` file following [these steps](https://github.com/inpsyde/playwright-utils?tab=readme-ov-file#env-variables). See also `.env.example`.
+
+8. Configure `playwright.config.ts` of the project following [these steps](https://github.com/inpsyde/playwright-utils?tab=readme-ov-file#playwright-configuration).
+
+9. Reporting. Add `testrail-reporter` in the reporter section of `playwright.config.ts`:
+
+	```ts
+	reporter: [
+		// other reporters ...
+		[ '@inpsyde/playwright-utils/build/integration/testrail/testrail-reporter.js' ],
+	],
+	```
+
+	Configure connection to TestRail API in `.env` (see `.env.example`):
+
+	```bash
+	# Testrail
+	TESTRAIL_URL=https://website.testrail.io
+	TESTRAIL_USERNAME=user@company.com
+	TESTRAIL_PASSWORD=*************
+	TESTRAIL_PLAN_ID=
+	TESTRAIL_RUN_ID=
+	```
+
+## Run tests
+
+To execute all tests sequentially, in the terminal navigate to the `./tests/qa` directory and run following command:
+
+```bash
+npx playwright test --project=all
+```
+
+### Additional options to run tests from command line
+
+- Add scripts to `package.json` of the project (eligible for Windows, not tested on other OS):
+
+	```json
+	"scripts": {
+		"all": "npx playwright test --project=all --workers=1",
+		"sequential": "npx playwright test --project=sequential --workers=1",
+		"parallel-surcharge": "npx playwright test --project=surcharge --workers=3"
+	},
+	```
+
+	Run script with the following command:
+
+	```bash
+	npm run parallel-transactions
+	```
+
+	\* - there's a number of tests which can be executed in parallel to speed up test execution (see `projects` section in `playwright.config.ts`).
+
+- Run several tests by test ID
+
+	- MacOS, Linux:
+
+	```bash
+	npx playwright test --grep "C123|C124|C125"
+	```
+
+	- Windows (Powershell):
+
+	```bash
+	npx playwright test --grep --% "C123^|C124^|C125"
+	```
+
+	It may be required additionally to specify the project (if tests relate to more then one project):
+
+	```bash
+	npx playwright test --project "project-name" --grep --% "C123^|C124^|C125"
+	```
+
+
+## Autotest Execution workflow
+
+1. Create test plan with run in TestRail, named after the tested plugin version, for example "Test Plan for Release 1.2.3".
+
+	\* - for autotest run there's no need to manually add tests cases to the run - the executed test will be added automatically before automated execution.
+
+2. Link release ticket (via `tests: JIR-234`).
+
+3. Set Test Execution ticket status `In progress`.
+
+4. Add/update test plan with run IDs in `.env` file of the project (`TESTRAIL_PLAN_ID, TESTRAIL_RUN_ID`).
+
+5. Download tested plugin `.zip` package (usually attached to release ticket) and add it to `/project/<project-name>/resources/files`. You may need to remove version number from the file name.
+
+6. Optional: delete previous version of tested plugin from the website if you don't execute __plugin foundation__ tests.
+
+7. Start autotest execution from command line for the defined scope of tests (e.g. all, Critical, etc.). You should see `Test plan ID: 001, Test run ID: 002` in the terminal.
+
+8. When finished test results should be exported to the specified test run ticket in Testrail.
+
+9. Analyze failed tests (if any). Restart execution for failed tests, possibly in debug mode (see section _Additional options to run tests from command line_):
+
+	```bash
+	npx playwright test --grep --% "C123^|C124^|C125" --debug
+	```
+
+10. Report bugs (if any) and attach them to the test-runs of failed tests.
+
+11. If needed fix failing tests in a new branch, create a PR and assign it for review.
+
+## Coding standards
+
+Before commiting changes run following command:
+
+```bash
+npm run lint:js:fix
+```

tests/qa/global-setup.ts

@@ -0,0 +1,29 @@
+/**
+ * External dependencies
+ */
+import { FullConfig } from '@playwright/test';
+import { restLogin, guestStorageState } from '@inpsyde/playwright-utils/build';
+
+async function globalSetup( config: FullConfig ) {
+	const projectUse = config.projects[ 0 ].use;
+
+	await restLogin( {
+		baseURL: projectUse.baseURL,
+		storageStatePath: String( projectUse.storageState ),
+		httpCredentials: projectUse.httpCredentials,
+		user: {
+			// @ts-ignore
+			username: process.env.WP_USERNAME,
+			// @ts-ignore
+			password: process.env.WP_PASSWORD,
+		},
+	} );
+
+	await guestStorageState( {
+		baseURL: projectUse.baseURL,
+		httpCredentials: projectUse.httpCredentials,
+		storageStatePath: `${ process.env.STORAGE_STATE_PATH }/guest.json`,
+	} );
+}
+
+export default globalSetup;

tests/qa/package.json

@@ -0,0 +1,62 @@
+{
+  "name": "mollie-playwright-tests",
+  "version": "1.0.0",
+  "description": "Playwright e2e tests for Mollie",
+  "main": "index.js",
+  "repository": {
+    "type": "git",
+    "url": "git+https://github.com/inpsyde/playwright-tests.git"
+  },
+  "author": {
+    "name": "Inpsyde GmbH",
+    "email": "hello@inpsyde.com",
+    "url": "https://inpsyde.com/"
+  },
+  "license": "GPL-3.0-or-later",
+  "bugs": {
+    "url": "https://github.com/inpsyde/playwright-tests/issues"
+  },
+  "homepage": "https://github.com/inpsyde/playwright-tests#readme",
+  "dependencies": {
+    "@inpsyde/playwright-utils": "^2.0.0-beta.2",
+    "dotenv": "^16.3.1",
+    "dotenv-cli": "^7.3.0",
+    "playwright": "^1.40.1",
+    "yarn": "^1.22.21"
+  },
+  "devDependencies": {
+    "@percy/cli": "^1.28.0",
+    "@percy/playwright": "^1.0.4",
+    "@playwright/test": "^1.38.1",
+    "@types/node": "^20.8.4",
+    "@wordpress/scripts": "^25.0.0"
+  },
+  "scripts": {
+    "setup:tests": "npm install && npx playwright install",
+    "setup:utils": "npm run setup:tests && cd ./playwright-utils && yarn devLocal",
+    "lint:md": "wp-scripts lint-md-docs ./**/*.md README.md",
+    "lint:js": "wp-scripts lint-js ./**/*.{ts,tsx,mjs} projects/**/.*/*.{ts,tsx,mjs}",
+    "lint:js:fix": "wp-scripts lint-js --fix ./**/*.{ts,tsx,mjs} ./**/**/*.{ts,tsx,mjs} ./**/.*/*.{ts,tsx,mjs} ./**/**/.*/*.{ts,tsx,mjs}",
+    "all": "npx playwright test --project=all --workers=1",
+    "sequential": "npx playwright test --project=sequential --workers=1",
+    "parallel-transaction-eur-classic": "npx playwright test --project=transaction-eur-classic --workers=5",
+    "parallel-transaction-eur-block": "npx playwright test --project=transaction-eur-block --workers=3",
+    "transaction-non-eur": "npx playwright test --project=transaction-non-eur workers=1",
+    "parallel-surcharge": "npx playwright test --project=surcharge --workers=3",
+    "parallel-transactions": "npm run parallel-transaction-eur-block & npm run parallel-transaction-eur-classic",
+    "all-with-parallel": "npm run sequential & npm run parallel-surcharge & npm run parallel-transactions"
+  },
+  "eslintConfig": {
+    "extends": [
+      "plugin:@wordpress/eslint-plugin/recommended"
+    ],
+    "rules": {
+      "@wordpress/dependency-group": "error",
+      "@wordpress/no-unsafe-wp-apis": "off",
+      "no-console": "off"
+    }
+  },
+  "workspaces": [
+    "playwright-utils"
+  ]
+}