feat: add useragent to conformance test run

[BobyMCbobs]

adds a per-test useragent containing gateway-api version and test features

/kind test
/area conformance

What this PR does / why we need it:

Which issue(s) this PR fixes:

Fixes #3193

NONE

[BobyMCbobs]

It would be really helpful if once this is merged, it can be backported to v1.0 and v1.1 test suites for historic data generation.

[youngnick]

Seems like this list could get pretty long, is it okay to have all the feature names present in every useragent string? How long can a useragent string be?

[BobyMCbobs]

@youngnick, good question!

Looking at the latest Kubernetes test results, filtering by conformance tests:

$ curl -sSL https://storage.googleapis.com/kubernetes-jenkins/logs/ci-kubernetes-e2e-gci-gce/1810176656531263488/artifacts/junit_01.xml \ 
  | yq -p=xml -o=yaml \
  | yq '.testsuites.testsuite.testcase[] | select(.+@name == "*[Conformance]*") | .+@name' \
  | sed 's/.*\]\(.*\)\[.*/\1/g' \
  | awk '{ print length($0) " " $0; }' | sort -r -n | cut -d ' ' -f 2- | head -n 1 | wc -c

The longest useragent is 131 characters.
The current longest useragent with the changes from this PR is

apisnoop=# select distinct(useragent) as ua, length(useragent) as ual from testing.audit_event where useragent ilike 'gateway-api-conformance.test%' order by ual desc limit 1;
                                                                         ua                                                                          |
 ual
-----------------------------------------------------------------------------------------------------------------------------------------------------+
-----
 gateway-api-conformance.test::v1.2.0-dev::HTTPRouteRequestMultipleMirrors::Gateway,HTTPRoute,HTTPRouteRequestMirror,HTTPRouteRequestMultipleMirrors |
 147
(1 row)

is 147.

Kubernetes Audit Logging appears to only store 1024 characters for useragents
https://github.com/kubernetes/kubernetes/blob/2b03f04/staging/src/k8s.io/apiserver/pkg/audit/request.go#L39

[robscott]

Thanks for the detailed analysis @BobyMCbobs! If I'm understanding this correctly, it seems like this would be easy to change in the future if/when we run into an issue, and the only limit that really matters is k8s API Server. Given that, I think I'm ok with the current path. In the future we could add a numerical id for each feature if needed.

[BobyMCbobs]

@robscott, absolutely!
Yes and the main thing is that we are able to map tests to features. Using this as a framework, we can pump any extra data through as usefulness and need arises.

[mlavacca]

In the future we could add a numerical id for each feature if needed.

I think this is a great idea, instead of putting all the features' names here.

[robscott]

Thanks @BobyMCbobs! Will defer LGTM to someone else. Unclear if we'll ever get to a v1.1.1 release, but this should at least create a cherry pick PR.

/cherry-pick release-1.1
/approve

[k8s-infra-cherrypick-robot]

@robscott: once the present PR merges, I will cherry-pick it on top of release-1.1 in a new PR and assign it to you.

In response to this:

Thanks @BobyMCbobs! Will defer LGTM to someone else. Unclear if we'll ever get to a v1.1.1 release, but this should at least create a cherry pick PR.

/cherry-pick release-1.1
/approve

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

[youngnick]

LGTM but I think that one of the conformance reviewers should check this before we merge. @mlavacca @LiorLieberman @sunjayBhatia maybe?

[sunjayBhatia]

LGTM in that this does not affect tests running as normal

(I did not verify the user agent was set in the api server audit logs myself however)

One question i have is if we would be missing any api server calls (or if it matters) since where setClientsetForTest is called happens when tests are run, after the base manifests are already applied in suite Setup:

gateway-api/conformance/utils/suite/suite.go

Lines 341 to 352 in 86c06a0

	tlog.Logf(t, "Test Setup: Applying base manifests")
	suite.Applier.MustApplyWithCleanup(t, suite.Client, suite.TimeoutConfig, suite.BaseManifests, suite.Cleanup)
	tlog.Logf(t, "Test Setup: Applying programmatic resources")
	secret := kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-web-backend", "certificate", []string{"*"})
	suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup)
	secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-infra", "tls-validity-checks-certificate", []string{"*", "*.org"})
	suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup)
	secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-infra", "tls-passthrough-checks-certificate", []string{"abc.example.com"})
	suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup)
	secret = kubernetes.MustCreateSelfSignedCertSecret(t, "gateway-conformance-app-backend", "tls-passthrough-checks-certificate", []string{"abc.example.com"})
	suite.Applier.MustApplyObjectsWithCleanup(t, suite.Client, suite.TimeoutConfig, []client.Object{secret}, suite.Cleanup)

[mlavacca]

We already have the config stored in the suite. Can we use that instead of re-creating it?

[BobyMCbobs]

Thank you for your comment - Good thinking, yes!

Made the change in 7a5e8df 5c3382f

[youngnick]

@BobyMCbobs, looks like this needs a rebase and has an outstanding comment, but is pretty close aside from those things.

[youngnick]

LGTM, but I think @mlavacca should be the last sign-off here.

[mlavacca]

Thanks, @BobyMCbobs!

/lgtm

[k8s-ci-robot]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: BobyMCbobs, mlavacca, robscott, sunjayBhatia

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• conformance/OWNERS [mlavacca,robscott,sunjayBhatia]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

[k8s-infra-cherrypick-robot]

@robscott: #3211 failed to apply on top of branch "release-1.1":

Applying: feat: add useragent to conformance test run
Using index info to reconstruct a base tree...
M	conformance/utils/suite/suite.go
Falling back to patching base and 3-way merge...
Auto-merging conformance/utils/suite/suite.go
CONFLICT (content): Merge conflict in conformance/utils/suite/suite.go
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 feat: add useragent to conformance test run
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".

In response to this:

Thanks @BobyMCbobs! Will defer LGTM to someone else. Unclear if we'll ever get to a v1.1.1 release, but this should at least create a cherry pick PR.

/cherry-pick release-1.1
/approve

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.