diff --git a/TESTING.md b/TESTING.md index 9c681936e7dc..5f5e54735c4f 100644 --- a/TESTING.md +++ b/TESTING.md @@ -6,17 +6,19 @@ Overview - [Unit tests](#unit-tests) - [Integration tests](#integration-tests) - [Functional tests](#functional-tests) + - [Backwards Compatibility tests](#backwards-compatibility-tests) - [Continuous Integration](#continuous-integration) - [Environment misc](#environment-misc) - [Misc](#misc) # General information -OpenSearch Dashboards uses [Jest](https://jestjs.io/) for unit and integration tests and [Selenium](https://www.selenium.dev/) for functional tests. +OpenSearch Dashboards uses [Jest](https://jestjs.io/) for unit and integration tests, [Selenium](https://www.selenium.dev/) for functional tests, and [Cypress](https://www.cypress.io/) for backwards compatibility tests. -In general, we recommend three tiers of tests: +In general, we recommend four tiers of tests: * Unit tests: Unit tests: small and modular tests that utilize mocks for external dependencies. * Integration tests: higher-level tests that verify interactions between systems (eg. HTTP APIs, OpenSearch API calls, calling other plugin). * End-to-end tests (e2e): functional tests that verify behavior in a web browser. +* Backwards Compatibility tests: cypress tests that verify any changes are backwards compatible with previous versions. # Requirements * Install the latest NodeJS, [NPM](https://www.npmjs.com/get-npm) and [Yarn](https://classic.yarnpkg.com/en/docs/install/#mac-stable) @@ -48,7 +50,20 @@ To debug functional tests: Say that you would want to debug a test in CI group 1, you can run the following command in your environment: `node --debug-brk --inspect scripts/functional_tests.js --config test/functional/config.js --include ciGroup1 --debug` -This will print of an address, to which you could open your chrome browser on your instance and navigate to `chrome://inspect/#devices` and inspect the functional test runner `scripts/functional_tests.js`. +This will print off an address, to which you could open your chrome browser on your instance and navigate to `chrome://inspect/#devices` and inspect the functional test runner `scripts/functional_tests.js`. + +### Backwards Compatibility tests +To run all the backwards compatibility tests on vanilla OpenSearch Dashboards: + +`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz]` + +To run all the backwards compatibility tests on bundled dashboards, pass the bundle parameter to the test: + +`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz] -b true` + +To run specific versions' backwards compatibility tests, pass the versions to the test: + +`yarn test:bwc -o [test path to opensearch.tar.gz] -d [test path to opensearch-dashboards.tar.gz] -v "[test versions]"` ### Additional checks Make sure you run lint checker before submitting a pull request. To run lint checker: @@ -70,5 +85,4 @@ By default the version of OpenSearch Dashboards will pull the snapshot of the sa Although Jest is the standard for this project, there are a few Mocha tests that still exist. You can run these tests by running: `yarn test:mocha` -However, these tests will eventually be migrated. Please avoid writing new Mocha tests. For further questions or to check the status please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215). - +However, these tests will eventually be migrated. Please avoid writing new Mocha tests. For further questions or to check the status please see this [issue](https://github.com/opensearch-project/OpenSearch-Dashboards/issues/215). \ No newline at end of file