Merge pull request #785 from magento/develop-master

Merge master into develop

Author: tomreece

.gitignore

@@ -19,4 +19,4 @@ dev/tests/mftf.log
 dev/tests/docs/*
 dev/tests/_output
 dev/tests/functional.suite.yml
-
+/v2/

docs/best-practices.md

@@ -37,11 +37,9 @@ We recommend to keep Action Groups having single responsibility, for example `Ad
 
 ## Contribute
 
-Althought the Magento Core team and Contributors join forces to cover most of the features with tests, it is impossible to have this done quickly.
+Although the Magento Core team and Contributors join forces to cover most of the features with tests, it is impossible to have this done quickly.
 If you've covered Magento Core feature with Functional Tests - you are more than welcome to contribute.
 
-You can also help with MFTF Test Migration to get the experience and valuable feedback from other community members and maintainers.
-
 ## Action group
 
 1. [Action group] names should be sufficiently descriptive to inform a test writer of what the action group does and when it should be used. Add additional explanation in annotations if needed.
@@ -108,7 +106,7 @@ Example: _StorefrontCreateCustomerTest.xml_.
 
 #### Action Group file name
 
-Format: {_Admin_ or _Storefront_}{Action Group Summary}ActionGroup.xml`, where Action Group Summary describes with a few words what we can expect from it.
+Format: {_Admin_ or _Storefront_}{Action Group Summary}ActionGroup.xml`, where Action Group Summary is a short description of what the action group does.
 
 Example: _AdminCreateStoreActionGroup.xml_ 
 
@@ -217,4 +215,3 @@ Since the configurable product module could be disabled, this approach is more r
 [merging]: merging.html
 [parameterized selectors]: section/parameterized-selectors.html
 [sections]: section.html
-[MFTF Test Migration]: https://github.com/magento/magento-functional-tests-migration

docs/commands/codeception.md

@@ -1,7 +1,7 @@
 # CLI commands: vendor/bin/codecept
 
 <div class="bs-callout bs-callout-warning" markdown="1">
-We do not recommend using Codeception commands directly as they can break the MFTF basic workflow.
+We do not recommend using Codeception commands directly as they can break MFTF basic workflow.
 All the Codeception commands you need are wrapped using the [mftf tool][].
 
 To run the Codeception testing framework commands directly, change your directory to the `<Magento root>`.
@@ -36,7 +36,7 @@ vendor/bin/codecept run
 ```
 
 <div class="bs-callout bs-callout-info">
-The following documentation corresponds to Codeception 2.3.8.
+The following documentation corresponds to Codeception 4.1.4.
 </div>
 
 ```bash
@@ -47,36 +47,44 @@ Arguments:
    test                  test to be run
 
 Options:
-   -o, --override=OVERRIDE Override config values (multiple values allowed)
-   --config (-c)         Use custom path for config
-   --report              Show output in compact style
-   --html                Generate html with results (default: "report.html")
-   --xml                 Generate JUnit XML Log (default: "report.xml")
-   --tap                 Generate Tap Log (default: "report.tap.log")
-   --json                Generate Json Log (default: "report.json")
-   --colors              Use colors in output
-   --no-colors           Force no colors in output (useful to override config file)
-   --silent              Only outputs suite names and final results
-   --steps               Show steps in output
-   --debug (-d)          Show debug and scenario output
-   --coverage            Run with code coverage (default: "coverage.serialized")
-   --coverage-html       Generate CodeCoverage HTML report in path (default: "coverage")
-   --coverage-xml        Generate CodeCoverage XML report in file (default: "coverage.xml")
-   --coverage-text       Generate CodeCoverage text report in file (default: "coverage.txt")
-   --coverage-phpunit    Generate CodeCoverage PHPUnit report in file (default: "coverage-phpunit")
-   --no-exit             Do not finish with exit code
-   --group (-g)          Groups of tests to be executed (multiple values allowed)
-   --skip (-s)           Skip selected suites (multiple values allowed)
-   --skip-group (-x)     Skip selected groups (multiple values allowed)
-   --env                 Run tests in selected environments. (multiple values allowed, environments can be merged with ',')
-   --fail-fast (-f)      Stop after first failure
-   --help (-h)           Display this help message.
-   --quiet (-q)          Do not output any message.
-   --verbose (-v|vv|vvv) Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
-   --version (-V)        Display this application version.
-   --ansi                Force ANSI output.
-   --no-ansi             Disable ANSI output.
-   --no-interaction (-n) Do not ask any interactive question.
+    -o, --override=OVERRIDE                    Override config values (multiple values allowed)
+    -e, --ext=EXT                              Run with extension enabled (multiple values allowed)
+        --report                               Show output in compact style
+        --html[=HTML]                          Generate html with results [default: "report.html"]
+        --xml[=XML]                            Generate JUnit XML Log [default: "report.xml"]
+        --phpunit-xml[=PHPUNIT-XML]            Generate PhpUnit XML Log [default: "phpunit-report.xml"]
+        --tap[=TAP]                            Generate Tap Log [default: "report.tap.log"]
+        --json[=JSON]                          Generate Json Log [default: "report.json"]
+        --colors                               Use colors in output
+        --no-colors                            Force no colors in output (useful to override config file)
+        --silent                               Only outputs suite names and final results
+        --steps                                Show steps in output
+    -d, --debug                                Show debug and scenario output
+        --bootstrap[=BOOTSTRAP]                Execute custom PHP script before running tests. Path can be absolute or relative to current working directory [default: false]
+        --no-redirect                          Do not redirect to Composer-installed version in vendor/codeception
+        --coverage[=COVERAGE]                  Run with code coverage
+        --coverage-html[=COVERAGE-HTML]        Generate CodeCoverage HTML report in path
+        --coverage-xml[=COVERAGE-XML]          Generate CodeCoverage XML report in file
+        --coverage-text[=COVERAGE-TEXT]        Generate CodeCoverage text report in file
+        --coverage-crap4j[=COVERAGE-CRAP4J]    Generate CodeCoverage report in Crap4J XML format
+        --coverage-phpunit[=COVERAGE-PHPUNIT]  Generate CodeCoverage PHPUnit report in path
+        --no-exit                              Don't finish with exit code
+    -g, --group=GROUP                          Groups of tests to be executed (multiple values allowed)
+    -s, --skip=SKIP                            Skip selected suites (multiple values allowed)
+    -x, --skip-group=SKIP-GROUP                Skip selected groups (multiple values allowed)
+        --env=ENV                              Run tests in selected environments. (multiple values allowed)
+    -f, --fail-fast                            Stop after first failure
+        --no-rebuild                           Do not rebuild actor classes on start
+        --seed=SEED                            Define random seed for shuffle setting
+        --no-artifacts                         Don't report about artifacts
+    -h, --help                                 Display this help message
+    -q, --quiet                                Do not output any message
+    -V, --version                              Display this application version
+        --ansi                                 Force ANSI output
+        --no-ansi                              Disable ANSI output
+    -n, --no-interaction                       Do not ask any interactive question
+    -c, --config[=CONFIG]                      Use custom path for config
+    -v|vv|vvv, --verbose                       Increase the verbosity of messages: 1 for normal output, 2 for more verbose output and 3 for debug
 ```
 
 <!-- Link definitions -->

docs/commands/mftf.md

@@ -45,7 +45,7 @@ vendor/bin/mftf generate:tests AdminLoginSuccessfulTest StorefrontPersistedCusto
 ### Generate test by test and suite name
 
 ```bash
-vendor/bin/mftf generate:tests LoginSuite:AdminLoginSuccessfulTest
+vendor/bin/mftf generate:tests WYSIWYGDisabledSuite:AdminCMSPageCreatePageTest
 ```
 
 ### Generate and run the tests for a specified group
@@ -62,15 +62,15 @@ This command cleans up the previously generated tests; generates and runs tests
 vendor/bin/mftf run:test AdminLoginSuccessfulTest StorefrontPersistedCustomerLoginTest -r
 ```
 
-This command cleans up the previously generated tests; generates and runs the `LoginAsAdminTest` and `LoginAsCustomerTest` tests.
+This command cleans up the previously generated tests; generates and runs the `AdminLoginSuccessfulTest` and `StorefrontPersistedCustomerLoginTest` tests.
 
 ### Generate and run particular test in a specific suite's context
 
 ```bash
-vendor/bin/mftf run:test LoginSuite:AdminLoginSuccessfulTest -r
+vendor/bin/mftf run:test WYSIWYGDisabledSuite:AdminCMSPageCreatePageTest -r
 ```
 
-This command cleans up previously generated tests; generates and run `AdminLoginSuccessfulTest` within the context of the `LoginSuite`.
+This command cleans up previously generated tests; generates and run `AdminCMSPageCreatePageTest` within the context of the `WYSIWYGDisabledSuite`.
 
 ### Generate and run a testManifest.txt file
 

docs/configuration.md

@@ -1,12 +1,12 @@
 # Configuration
 
 The `*.env` file provides additional configuration for the Magento Functional Testing Framework (MFTF).
-To run the MFTF on your Magento instance, specify the basic configuration values.
+To run MFTF on your Magento instance, specify the basic configuration values.
 Advanced users can create custom configurations based on requirements and environment.
 
 ## Basic configuration
 
-These basic configuration values are __required__ and must be set by the user before the MFTF can function correctly.
+These basic configuration values are __required__ and must be set by the user before MFTF can function correctly.
 
 ### MAGENTO_BASE_URL
 
@@ -34,7 +34,7 @@ MAGENTO_BACKEND_NAME=admin_12346
 
 ### MAGENTO_BACKEND_BASE_URL
 
-(Optional) If you are running the Admin Panel on separate a domain, specify this value:
+(Optional) If you are running the Admin Panel on a separate domain, specify this value:
 
 Example:
 
@@ -64,7 +64,7 @@ MAGENTO_ADMIN_PASSWORD=1234reTyt%$7
 
 ## Advanced configuration
 
-Depending on the environment you use, you may need to configure the MFTF more precisely by setting more configuration parameters then for basic configuration.
+Depending on the environment you use, you may need to configure MFTF more precisely by setting additional configuration parameters.
 This section describes available configuration parameters and their default values (where applicable).
 
 ### DEFAULT_TIMEZONE
@@ -173,8 +173,8 @@ MAGENTO_RESTAPI_SERVER_PORT=5000
 ### \*_BP
 
 Settings to override base paths for the framework.
-You can use it when the MFTF is applied as a separate tool.
-For example, when you need to place the MFTF and the Magento codebase in separate projects.
+You can use it when MFTF is applied as a separate tool.
+For example, when you need to place MFTF and the Magento codebase in separate projects.
 
 ```conf
 MAGENTO_BP
@@ -221,13 +221,13 @@ The path to where the MFTF modules mirror Magento modules.
 Example:
 
 ```conf
-TESTS_MODULE_PATH=~/magento2/dev/tests/acceptance/tests/functional/Magento/FunctionalTest
+TESTS_MODULE_PATH=~/magento2/dev/tests/acceptance/tests/functional/Magento
 ```
 
 ### MODULE_ALLOWLIST
 
 Use for a new module.
-When adding a new directory at `Magento/FunctionalTest`, add the directory name to `MODULE_ALLOWLIST` to enable the MFTF to process it.
+When adding a new directory at `tests/functional/Magento`, add the directory name to `MODULE_ALLOWLIST` to enable MFTF to process it.
 
 Example:
 
@@ -245,7 +245,7 @@ It points to `MAGENTO_BASE_URL` + `dev/tests/acceptance/utils/command.php`
 Modify the default value:
 
 -  for non-default Magento installation
--  when use a subdirectory in the `MAGENTO_BASE_URL`
+-  when using a subdirectory in the `MAGENTO_BASE_URL`
 
 Example: `dev/tests/acceptance/utils/command.php`
 

docs/configure-2fa.md

@@ -44,6 +44,10 @@ Use the action `getOTP` [Reference](./test/actions.md#getotp) to generate the co
 Note:
 You will need to set the `secret` for any non-default admin users first, before using `getOTP`. For example:
 
+{%raw%}
+
 ```xml
 <magentoCLI command="security:tfa:google:set-secret admin2 {{_CREDS.magento/tfa/OTP_SHARED_SECRET}}" stepKey="setSecret"/>
 ```
+
+{%endraw%}

docs/credentials.md

@@ -3,20 +3,20 @@
 When you test functionality that involves external services such as UPS, FedEx, PayPal, or SignifyD,
 use the MFTF credentials feature to hide sensitive [data][] like integration tokens and API keys.
 
-Currently the MFTF supports three types of credential storage:
+Currently MFTF supports three types of credential storage:
 
 -  **.credentials file**
 -  **HashiCorp Vault**
 -  **AWS Secrets Manager**
 
 ## Configure File Storage
 
-The MFTF creates a sample file for credentials during [initial setup][]: `magento2/dev/tests/acceptance/.credentials.example`.
+MFTF creates a sample file for credentials during [initial setup][]: `magento2/dev/tests/acceptance/.credentials.example`.
 The file contains an example list of keys for fields that can require credentials.
 
 ### Create `.credentials`
 
-To make the MFTF process the file with credentials, in the command line, navigate to `magento2/dev/tests/acceptance/` and rename `.credentials.example` to `.credentials`.
+To make MFTF process the file with credentials, in the command line, navigate to `magento2/dev/tests/acceptance/` and rename `.credentials.example` to `.credentials`.
 
 ```bash
 cd dev/tests/acceptance/
@@ -78,7 +78,7 @@ vendor/my_awesome_service_token=rRVSVnh3cbDsVG39oTMz4A
 Hashicorp vault secures, stores, and tightly controls access to data in modern computing.
 It provides advanced data protection for your testing credentials.
 
-The MFTF works with both `vault enterprise` and `vault open source` that use `KV Version 2` secret engine.
+MFTF works with both `vault enterprise` and `vault open source` that use `KV Version 2` secret engine.
 
 ### Install vault CLI
 
@@ -92,11 +92,11 @@ Authenticate to vault server via the vault CLI tool: [Login Vault][Login Vault].
 vault login -method -path
 ```
 
-**Do not** use `-no-store` command option, as the MFTF will rely on the persisted token in the token helper (usually the local filesystem) for future API requests.
+**Do not** use `-no-store` command option, as MFTF will rely on the persisted token in the token helper (usually the local filesystem) for future API requests.
 
 ### Store secrets in vault
 
-The MFTF uses the `KV Version 2` secret engine for secret storage.
+MFTF uses the `KV Version 2` secret engine for secret storage.
 More information for working with `KV Version 2` can be found in [Vault KV2][Vault KV2].
 
 #### Secrets path and key convention
@@ -225,7 +225,7 @@ export CREDENTIAL_AWS_ACCOUNT_ID=<Your_12_Digits_AWS_Account_ID>
 ## Configure multiple credential storage
 
 It is possible and sometimes useful to setup and use multiple credential storage at the same time.
-In this case, the MFTF tests are able to read secret data at runtime from all storage options, in this case MFTF use the following precedence:
+In this case, the MFTF tests are able to read secret data at runtime from all storage options. MFTF will use the following precedence:
 
 ```
 .credentials File > HashiCorp Vault > AWS Secrets Manager
@@ -253,7 +253,7 @@ For example, to reference secret data in the [`fillField`][] action, use the `us
 ## Implementation details
 
 The generated tests do not contain credentials values.
-The MFTF dynamically retrieves, encrypts, and decrypts the sensitive data during test execution.
+MFTF dynamically retrieves, encrypts, and decrypts the sensitive data during test execution.
 Decrypted credentials do not appear in the console, error logs, or [test reports][].
 The decrypted values are only available in the `.credentials` file or within vault.
 
@@ -277,4 +277,4 @@ The MFTF tests delivered with Magento application do not use credentials and do
 [`CREDENTIAL_AWS_SECRETS_MANAGER_PROFILE`]: configuration.md#credential_aws_secrets_manager_profile
 [`CREDENTIAL_AWS_SECRETS_MANAGER_REGION`]: configuration.md#credential_aws_secrets_manager_region
 [Key Management Service]: https://aws.amazon.com/kms/
-[Amazon Resource Name]: https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html
+[Amazon Resource Name]: https://docs.aws.amazon.com/general/latest/gr/aws-arns-and-namespaces.html