Gatsby Testing Starter

Kick off your project with this default boilerplate. This starter ships with configured testing frameworks and tools for each layer of the Test Pyramid and more.

• Unit Testing: Jest with React Testing Library

• Structural Testing: Jest Snapshot Testing

• End-to-End Testing: Cypress with Cypress Testing Library

• Accessibility Testing: axe with cypress-axe

• Automated Visual Testing:

  • Storybook with jest-puppeteer and jest-image-snapshot
  • Cypress with Cypress Image Snapshot

Contents

• 🚀 Quick start
• ✅ Writing tests
• ✅ Automated Visual Testing
• ⚙️ GitHub Actions integration
• ⚙️ Netlify integration
• ⚙️ Codecov integration
• ⚙️ FOSSA integration
• 🧐 What's inside?

🚀 Quick start

1. Create a Gatsby site.

   Use the Gatsby CLI to create a new site, specifying the starter.

   npm install -g gatsby-cli
   gatsby new my-testing-starter https://github.com/DanailMinchev/gatsby-starter-testing

   or

   npx --force gatsby new my-testing-starter https://github.com/DanailMinchev/gatsby-starter-testing

2. Install dependencies.

   Navigate into your new site's directory and install dependencies.

   cd my-testing-starter/
   npm ci

3. Run unit and structural tests.

   After installing dependencies using npm ci, you can run the unit and structural tests in your site's directory.

   npm run test

   or you can run them in "watch" mode:

   npm run test:watch

   and you can update the Jest snapshots:

   npm run test:update

4. Run e2e (end-to-end), accessibility and visual tests using Cypress.

   After installing dependencies using npm ci, you need to build the project to run e2e tests in ci mode.

   npm run build

   Then, you can run the tests using test:e2e which will run in ci mode:

   npm run test:e2e

   Corresponding option to update image snapshots in this mode:

   npm run test:e2e:update

   You can run e2e tests explicitly in dev mode with hot reloading:

   npm run test:e2e:dev

   Corresponding option to update image snapshots in this mode:

   npm run test:e2e:dev:update

   You can run e2e tests explicitly in ci mode, the same way as it runs on your Continuous Integration (CI) platform:

   npm run test:e2e:ci

   Corresponding option to update image snapshots in this mode:

   npm run test:e2e:ci:update

5. Run e2e (end-to-end), accessibility and visual tests using Cypress running in Docker.

   You should have XQuartz installed and configured as described in Running GUI applications using Docker for Mac in case you would like to use Cypress Test Runner running in Docker on macOS.

   To run Cypress in Docker you can use similar commands as in point 4, but append :docker in the end.

   You can run the tests using test:e2e:docker which will run in ci mode.

   The test:e2e:docker command is working only on macOS and before running it you should run . docker/setupXQuartz.sh:

   . docker/setupXQuartz.sh
   npm run test:e2e:docker

   Corresponding option to update image snapshots in this mode:

   . docker/setupXQuartz.sh
   npm run test:e2e:docker:update

   You can run e2e tests explicitly in dev mode with hot reloading.

   The test:e2e:dev:docker command is working only on macOS and before running it you should run . docker/setupXQuartz.sh:

   . docker/setupXQuartz.sh
   npm run test:e2e:dev:docker

   Corresponding option to update image snapshots in this mode:

   . docker/setupXQuartz.sh
   npm run test:e2e:dev:docker:update

   You can run e2e tests explicitly in ci mode, the same way as it runs on your Continuous Integration (CI) platform:

   npm run test:e2e:ci:docker

   Corresponding option to update image snapshots in this mode:

   npm run test:e2e:ci:docker:update

6. Run automated visual tests.

   After installing dependencies using npm ci, you need to build the project to run Storybook and visual tests.

   npm run build

   Then, you can run the tests using test:visual which will start Storybook and Puppeteer:

   npm run test:visual

   and you can update the Jest image snapshots:

   npm run test:visual:update

7. Run automated visual tests in Docker.

   To run automated visual tests in Docker you can use similar commands as in point 6, but append :docker in the end.

   You can run the tests using test:visual:docker which will start Storybook and Puppeteer in Docker:

   npm run test:visual:docker

   and you can update the Jest image snapshots:

   npm run test:visual:docker:update

8. Start developing.

   Navigate into your new site's directory and start the app using develop or start script, or if you prefer you can start it together with e2e tests in watch mode:

   cd my-testing-starter/
   npm run test:e2e:dev

   and you can start unit tests in watch mode in another terminal:

   npm run test:watch

9. Open the source code and start editing!

   Your site is now running at http://localhost:8000/!

   Open the my-testing-starter directory in your code editor of choice and edit src/pages/index.js. Save your changes and the browser will update in real time. The unit tests will re-run automatically and Cypress will reload the app so that you can re-trigger the Cypress tests.

10. Git Hooks.

    This project is using git hooks which are configured with Husky.

    To enable Git hooks, install Husky manually:

    npx husky install

    "pre-commit" git hook

    This hook is configured to format your code with Prettier and lint-staged. When new files are staged and committed they will be formatted same way as npm run format command.

    "commit-msg" git hook

    Your commit messages must be compatible with Conventional Commits specification. commitlint is configured to make sure this specification is enforced. You can use npm run commit command to commit your changes which is using prompt to create the commit messages.

✅ Writing tests

There are few conventions when writing tests.

1. Automated Visual Testing and Storybook.

   You can write your Storybook stories in /stories/ directory and later you can write visual tests in __visual_tests__/ directories. More information can be found here.

✅ Automated Visual Testing

There are two options for Automated Visual Testing:

• Storybook with jest-puppeteer and jest-image-snapshot
• Cypress with Cypress Image Snapshot

When creating the screenshots (image snapshots) there might be differences in font size or other platform specific rendering in the browser when running locally and on CI.

That is why it is better to run the visual tests inside Docker, so that the same platform and browser is used locally and on CI.

For macOS users there is an option to run Cypress Test Runner inside Docker as well.

Please see following articles:

• Running GUI applications using Docker for Mac
• Run Cypress with a single Docker command

⚙️ GitHub Actions integration

There are currently 3 GitHub Actions workflows:

• CI This workflow is acting as CI pipeline.
  It is using main_build_environment environment (needs to be created).

• Pull Request This workflow runs when a new Pull Request is created or modified. It is similar to CI workflow.
  It is using pr_build_environment environment (needs to be created).
  The pr_build_environment environment can be setup to have required reviewers, so that the build is not triggered automatically.

• Deploy This workflow can be used to deploy latest master branch or specific commit to Netlify. Use this to deploy to preview, uat, prod.

• Release This workflow can be used to create new release automatically.

  It will run all the tests as in CI workflow, it will create git release tag, new GitHub release and deploy to Netlify's UAT url.

  Make sure to edit package.json and repository.url property so that semantic-release works properly.

⚙️ Netlify integration

This app defines Netlify configuration in netlify.toml file.

Currently, the build and deploy process is done via GitHub Actions integration instead of Netlify Build.

Netlify Edge is used to host the resources which are build and deployed by GitHub Actions integration.

Netlify configuration

Connect the repository to Netlify by following Deploy with Git documentation.

You need to stop Netlify builds as described here.

Netlify GitHub configuration

You need to configure GitHub encrypted secrets for GitHub Actions to be able to deploy the app.

Follow the steps below:

1. Register a new Netlify personal access token as described in Obtain a token in the Netlify UI document. Copy the value.
2. Register a new GitHub encrypted secret as described in Creating encrypted secrets for a repository.
   • Name: NETLIFY_AUTH_TOKEN
   • Value: the personal access token value from step 1.
3. Register a new GitHub encrypted secret as described in Creating encrypted secrets for a repository.
   • Name: NETLIFY_SITE_ID
   • Value: from the Netlify site dashboard, go to Settings > General > Site details > Site information, and copy the value. More information here.

Netlify deployments - GitHub Actions

Please see GitHub Actions integration section for more information about the available options.

Netlify deployments - Local machine

To set up your local machine with Netlify (deployments and live dev) follow the steps below:

1. Copy .env.example file and name it .env:

   cp .env.example .env
   

2. Register a new Netlify personal access token as described in Obtain a token in the Netlify UI document. Copy the value.
3. Edit the .env file and add the token from step 2 to the NETLIFY_AUTH_TOKEN variable.

   NETLIFY_AUTH_TOKEN=your-token-value-from-step-2-here
   

4. From the Netlify site dashboard, go to Settings > General > Site details > Site information, and copy the API ID value as described in Link with an environment variable.
5. Edit the .env file and add the app id value from step 4 to the NETLIFY_SITE_ID variable.

   NETLIFY_SITE_ID=your-app-id-value-from-step-4-here
   

Now you should be able to interact with Netlify platform from your local machine.

You can use following npm scripts with Netlify:

• npm run develop:netlify This will run local Netlify server on http://localhost:8888.
• npm run develop:netlify:live This will run local Netlify server with Live Share.
• npm run deploy:preview This will deploy the current local build to Draft url.
• npm run deploy:uat This will deploy the current local build to UAT url.
• npm run deploy:prod This will deploy the current local build to PRODUCTION url.

⚙️ Codecov integration

Follow the steps below:

1. Copy the Upload token as described in the documentation. Copy the value.
2. Register a new GitHub encrypted secret as described in Creating encrypted secrets for a repository.
   • Name: CODECOV_TOKEN
   • Value: the personal access token value from step 1.

You can configure Codecov in codecov.yml file.

⚙️ FOSSA integration

Please follow official documentation.

🧐 What's inside?

A quick look at the top-level files and directories you'll see in a Gatsby project.

.
├── .cache
├── .circleci
├── .github
├── .storybook
├── __mocks__
├── coverage
├── cypress
├── docker
├── node_modules
├── public
├── src
├── static
├── stories
├── storybook-static
├── .dockerignore
├── .gitattributes
├── .gitignore
├── .npmrc
├── .nvmrc
├── .prettierignore
├── .prettierrc
├── app.Dockerfile
├── CODE_OF_CONDUCT.md
├── commitlint.config.js
├── CONTRIBUTING.md
├── cypress.json
├── gatsby-browser.js
├── gatsby-config.js
├── gatsby-node.js
├── gatsby-ssr.js
├── jest.config.js
├── jest.setup.js
├── jest.visual.config.js
├── jest-loadershim.js
├── jest-preprocess.js
├── jest-puppeteer.config.js
├── LICENSE
├── package.json
├── package-lock.json
├── README.md
└── test.Dockerfile

1. .cache: This directory is autogenerated. This directory is internal to Gatsby used for caching.

2. .circleci: This directory is used by CircleCI to build and test the project. The pipeline is defined in .circleci/config.yml file.

3. .github: This directory is used by GitHub. Currently, it has Dependabot and GitHub Actions files.

4. .storybook: This directory is used by Storybook to store the configuration and setup as described here.

5. __mocks__: This directory is used by Jest to store various mocks as described here.

6. coverage: This directory is autogenerated. This directory is generated by Jest when running the tests and has test coverage reports.

7. cypress: This directory is used by Cypress to store Cypress tests, fixtures, plugins and test artifacts (Cypress screenshots and videos) as described here. The Cypress tests are located in /cypress/e2e directory.

8. docker: This directory is used by Docker Compose to run Docker containers used mainly for Automated Visual Testing.

9. node_modules: This directory is autogenerated when you run npm run ci. This directory contains all of the modules of code that your project depends on (npm packages) are automatically installed.

10. public: This directory is autogenerated when you run npm run build. This directory contains all of your application code and assets which can be deployed to production.

11. src: This directory will contain the source code of your application.

12. static: This directory will contain various assets which will be added to public/ directory automatically when build the project. More information can be found here.

13. stories: This directory will contain various stories for your application used by Storybook as well as used for Automated Visual Testing.

14. storybook-static: This directory is autogenerated when you run npm run build-storybook. This directory will contain Storybook application shipped with various stories. The directory can be shared or deployed within the team.

15. .dockerignore: This file tells Docker which files it should not send to Docker daemon when building the Docker images.

16. .gitattributes: This file is used by Git to define attributes to pathnames.

17. .gitignore: This file tells Git which files it should not track / not maintain a version history for.

18. .npmrc: This file is used by npm to set npm options.

19. .nvmrc: This file is used by nvm to use the correct Node.js version for this application.

20. .prettierignore: This file tells Prettier which files it should not format.

21. .prettierrc: This is a configuration file for Prettier. Prettier is a tool to help keep the formatting of your code consistent.

22. app.Dockerfile: This file is used by Docker and defines the Docker image for the app.

23. CODE_OF_CONDUCT.md: "Contributor Code of Conduct" document.

24. commitlint.config.js: This is a configuration file for commitlint.

25. CONTRIBUTING.md: "Contributing" document.

26. cypress.json: This is a configuration file for Cypress. More information can be found here.

27. gatsby-browser.js: This file is where Gatsby expects to find any usage of the Gatsby browser APIs (if any). These allow customization/extension of default Gatsby settings affecting the browser.

28. gatsby-config.js: This is the main configuration file for a Gatsby site. This is where you can specify information about your site (metadata) like the site title and description, which Gatsby plugins you'd like to include, etc. (Check out the config docs for more detail).

29. gatsby-node.js: This file is where Gatsby expects to find any usage of the Gatsby Node APIs (if any). These allow customization/extension of default Gatsby settings affecting pieces of the site build process.

30. gatsby-ssr.js: This file is where Gatsby expects to find any usage of the Gatsby server-side rendering APIs (if any). These allow customization of default Gatsby settings affecting server-side rendering.

31. jest.config.js: This is a configuration file for Jest and it is used when you run npm run test for Unit Testing and Structural Testing. More information can be found here and here.

32. jest.setup.js: This is a setup file for Jest that runs some code to configure or set up the testing framework before each test file in the suite while it is executed. More information can be found here.

33. jest.visual.config.js: This is a configuration file for Jest and it is used when you run npm run test:visual for Automated Visual Testing. More information can be found here and here.

34. jest-loadershim.js: This is a setup file for Jest and it is used to configure or set up the testing environment. More information can be found here.

35. jest-preprocess.js: This is a setup file for Jest that defines transformers. More information can be found here.

36. jest-puppeteer.config.js: This is a configuration file for jest-puppeteer and it is used when you run npm run test:visual for Automated Visual Testing. More information can be found here.

37. LICENSE: Gatsby is licensed under the MIT license.

38. package.json: A manifest file for Node.js projects, which includes things like metadata (the project's name, author, etc). This manifest is how npm knows which packages to install for your project.

39. package-lock.json (See package.json above, first). This is an automatically generated file based on the exact versions of your npm dependencies that were installed for your project. (You won't change this file directly).

40. README.md: A text file containing useful reference information about your project.

41. test.Dockerfile: This file is used by Docker and defines the Docker image for Automated Visual Testing.