Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[CI] [Don't merge yet] Integration tests regrouping #6349

Open
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

pazone
Copy link
Contributor

@pazone pazone commented Dec 17, 2024

What does this PR do?

Extracts 3 new test groups that are executed separately:

  • standalone-upgrade
  • fleet-endpoint-security
  • install-uninstall (our favorite)

Now the longest running group is fleet-endpoint-security (42 minutes)

The TestRepeatedInstallUninstall on PRs will have 50 iterations instead of 100. For the main and release branches the iterations number will stay 100.

Why is it important?

Checklist

  • My code follows the style guidelines of this project
  • I have commented my code, particularly in hard-to-understand areas
  • I have made corresponding changes to the documentation
  • I have made corresponding change to the default configuration files
  • I have added tests that prove my fix is effective or that my feature works
  • I have added an entry in ./changelog/fragments using the changelog tool
  • I have added an integration test or an E2E test

Disruptive User Impact

How to test this PR locally

Related issues

Questions to ask yourself

  • How are we going to support this in production?
  • How are we going to measure its adoption?
  • How are we going to debug this?
  • What are the metrics I should take care of?
  • ...

testing/integration/install_test.go Outdated Show resolved Hide resolved
testing/integration/install_test.go Outdated Show resolved Hide resolved
@pazone pazone marked this pull request as ready for review December 17, 2024 22:02
@pazone pazone requested review from a team as code owners December 17, 2024 22:02
@pazone pazone requested review from swiatekm and pchila December 17, 2024 22:02
@pazone pazone marked this pull request as draft December 17, 2024 22:31
@pazone pazone changed the title [CI] TestRepeatedInstallUninstallFleet optimisation [CI] Integration tests regrouping Dec 17, 2024
@pazone pazone added backport-8.x Automated backport to the 8.x branch with mergify backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify and removed backport-skip labels Dec 18, 2024
@pazone pazone marked this pull request as ready for review December 18, 2024 10:31
Comment on lines +502 to +504
if os.Getenv("BUILDKITE_PULL_REQUEST") != "" {
return 50
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This deserves a comment in my opinion.

@@ -40,8 +40,11 @@ steps:
matrix:
- default
- fleet
- fleet-endpoint-security
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The ordering in these matrix lists looks arbitrary. Is there a reason for the ordering, or should we have them sorted alphabetically?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Honestly, there are no ordering rules. Just placed fleet groups together

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you going to sort them? If the order doesn't matter, I think it'd be a better user experience when engineers are scrolling through the results of a BK job to have them alphabetical. Plus it would make these lists more easily maintainable (until we figure out making the groups/matrix dynamic).

Copy link
Contributor

@rowlandgeoff rowlandgeoff left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Once the 2 open comments are addressed, LGTM

Copy link

@rowlandgeoff
Copy link
Contributor

I think this PR is now blocked until the issue listed here is understood? https://github.com/elastic/ingest-dev/issues/4216#issuecomment-2554151178

@pazone pazone changed the title [CI] Integration tests regrouping [CI] [Don't merge yet] Integration tests regrouping Dec 23, 2024
@pazone
Copy link
Contributor Author

pazone commented Dec 23, 2024

Blocked by this issue: https://github.com/elastic/ingest-dev/issues/4610

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
backport-8.x Automated backport to the 8.x branch with mergify backport-8.16 Automated backport with mergify backport-8.17 Automated backport with mergify enhancement New feature or request skip-changelog Team:Ingest-EngProd
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants