Skip to content

End to end and integration tests should be run for a controlled matrix of compiler releases #80

@euanh

Description

@euanh

The end to end and integration tests use the Swift compiler provided by the GitHub CI runner. The runner image was updated to include Swift 6.1 (https://github.com/actions/runner-images/blob/ubuntu24/20250406.1/images/ubuntu/Ubuntu2404-Readme.md) but this caused the end to end and integration tests to fail because they install the 6.0.2 release of the Static Linux SDK, which is not compatible with Swift 6.1.

#79 fixes this problem in the short term by updating the workflows to a compatible version of the Static Linux SDK, but ideally these tests should not depend on a the particular version of Swift in a runner image, and should also test multiple versions.

These tests do not run in containers because part of the test involves building and running a container image, which is inconvenient when the test itself runs in a container, as used for the matrix unit tests.

Possible solutions:

  • run the end to end and integration tests in privileged containers and pass the Docker socket through so images can be built and run
  • run the tests in Docker-in-Docker, which can make it easier to set up and tear down such recursive containers.
  • Use named runner images with known Swift compilers instead of ubuntu-latest.
  • Install specific compilers using swiftly.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions