Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

conformance profiles CLI #2066

Merged
merged 4 commits into from
Jul 11, 2023
Merged

conformance profiles CLI #2066

merged 4 commits into from
Jul 11, 2023

Conversation

mlavacca
Copy link
Member

@mlavacca mlavacca commented May 26, 2023 •
edited
Loading

What type of PR is this?

/kind feature
/area conformance

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #1990

Does this PR introduce a user-facing change?:

The experimental conformance profile suite can now be added as a stand-alone cli and by means of `go test`.

@k8s-ci-robot k8s-ci-robot added kind/feature Categorizes issue or PR as related to a new feature. area/conformance do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels May 26, 2023
@mlavacca mlavacca marked this pull request as draft May 26, 2023 15:43
@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label May 26, 2023
@k8s-ci-robot k8s-ci-robot added do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. size/M Denotes a PR that changes 30-99 lines, ignoring generated files. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. and removed size/M Denotes a PR that changes 30-99 lines, ignoring generated files. labels May 26, 2023
@k8s-ci-robot k8s-ci-robot added needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. labels Jun 1, 2023
shaneutt
shaneutt reviewed Jun 6, 2023
View reviewed changes
cmd/certification/cmd/config.go Outdated Show resolved Hide resolved
cmd/certification/cmd/config.go Outdated Show resolved Hide resolved
cmd/certification/cmd/root.go Outdated Show resolved Hide resolved
cmd/certification/cmd/root.go Outdated Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added size/L Denotes a PR that changes 100-499 lines, ignoring generated files. and removed needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Jul 7, 2023
@mlavacca mlavacca force-pushed the conformance-profiles-cli branch 7 times, most recently from 8bd8045 to 1888eed Compare July 7, 2023 12:28
@mlavacca mlavacca requested a review from shaneutt July 7, 2023 12:29
@mlavacca mlavacca marked this pull request as ready for review July 7, 2023 12:30
@k8s-ci-robot k8s-ci-robot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jul 7, 2023
@k8s-ci-robot k8s-ci-robot added release-note Denotes a PR that will be considered when it comes time to generate release notes. and removed do-not-merge/release-note-label-needed Indicates that a PR should not merge because it's missing one of the release note labels. labels Jul 7, 2023
@shaneutt
Copy link
Member

shaneutt commented Jul 7, 2023

/cc @sunjayBhatia @arkodg

Would appreciate your reviews of this as well if you have some time.

@arkodg
Copy link
Contributor

arkodg commented Jul 7, 2023 •
edited
Loading

hey @mlavacca, this looks good !
would be great if you could share the o/p snippet when run against any implementation

@mlavacca
feat: conformance profiles cli scaffolding
5d201da 
Signed-off-by: Mattia Lavacca <lavacca.mattia@gmail.com>
@mlavacca
feat: conformance profiles cli
bf10ef6 
experimental conformance tests added to the already existing conformance
test suite.

Signed-off-by: Mattia Lavacca <lavacca.mattia@gmail.com>
@mlavacca mlavacca force-pushed the conformance-profiles-cli branch 2 times, most recently from 45b9706 to dcef337 Compare July 10, 2023 09:14
@mlavacca
chore: refactor to improve readability
af424c1 
Signed-off-by: Mattia Lavacca <lavacca.mattia@gmail.com>
@mlavacca
Copy link
Member Author

hey @mlavacca, this looks good ! would be great if you could share the o/p snippet when run against any implementation

Here is the makefile command to run the experimental suite:

CONFORMANCE_FLAGS="--conformance-profiles=HTTP,TLS --organization=acme.org --project=test --url=acme.org/test --version=0.1.0 --contact=@johndoe" make conformance.experimental

And in this PR description there is a sample of the test.

@shaneutt shaneutt requested a review from arkodg July 10, 2023 12:49
shaneutt
shaneutt approved these changes Jul 10, 2023
View reviewed changes
Copy link
Member

@shaneutt shaneutt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great! Thanks for kicking this one out @mlavacca 🥳

I have no hard blockers:

/approve
/lgtm

I do have a few comments that I would like to see resolved before we merge it, and also want to make sure @arkodg gets a chance to complete his review so let's not merge until those are done:

/hold

conformance/experimental_conformance_test.go Outdated Show resolved Hide resolved
conformance/experimental_conformance_test.go Outdated Show resolved Hide resolved
conformance/utils/flags/experimental_flags.go Outdated Show resolved Hide resolved
conformance/utils/suite/experimental_suite.go Outdated Show resolved Hide resolved
conformance/utils/suite/experimental_suite.go Show resolved Hide resolved
@k8s-ci-robot k8s-ci-robot added the do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. label Jul 10, 2023
@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. approved Indicates a PR has been approved by an approver from all required OWNERS files. labels Jul 10, 2023
arkodg
arkodg approved these changes Jul 10, 2023
View reviewed changes
Copy link
Contributor

@arkodg arkodg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM thanks !

@mlavacca mlavacca mentioned this pull request Jul 11, 2023
Open
@mlavacca
chore: address reviews' comments
a4e8bad 
Signed-off-by: Mattia Lavacca <lavacca.mattia@gmail.com>
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 11, 2023
shaneutt
shaneutt approved these changes Jul 11, 2023
View reviewed changes
Copy link
Member

@shaneutt shaneutt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @mlavacca!

/approve
/lgtm
/unhold

@k8s-ci-robot k8s-ci-robot added lgtm "Looks good to me", indicates that a PR is ready to be merged. and removed do-not-merge/hold Indicates that a PR should not merge because someone has issued a /hold command. labels Jul 11, 2023
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: arkodg, mlavacca, shaneutt

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit 22c7e2a into kubernetes-sigs:main Jul 11, 2023
@pleshakov pleshakov mentioned this pull request Jul 13, 2023
Merged
6 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@shaneutt shaneutt shaneutt approved these changes

@arkodg arkodg arkodg approved these changes

@bowei bowei Awaiting requested review from bowei

@michaelbeaumont michaelbeaumont Awaiting requested review from michaelbeaumont

@sunjayBhatia sunjayBhatia Awaiting requested review from sunjayBhatia

Assignees

@shaneutt shaneutt

Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. area/conformance cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. kind/feature Categorizes issue or PR as related to a new feature. lgtm "Looks good to me", indicates that a PR is ready to be merged. release-note Denotes a PR that will be considered when it comes time to generate release notes. size/L Denotes a PR that changes 100-499 lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

CLI for the experimental version of the conformance test suite
4 participants
@mlavacca @shaneutt @arkodg @k8s-ci-robot