Store E2E manifests to artifacts directory

[tenzen-y]

What this PR does / why we need it:
I added a shell script to generate E2E dedicated manifests and store them in the artifacts directory.
It would be better not to modify production manifests.

Which issue(s) this PR fixes (optional, in Fixes #<issue number>, #<issue number>, ... format, will close the issue(s) when PR gets merged):
Fixes #

Checklist:

• Docs included if any changes are user facing

[andreyvelich]

Great addition!
Let's also modify the path for output Notebooks to not upload manifests to the GitHub:
https://github.com/kubeflow/trainer/blob/master/.github/workflows/test-e2e.yaml#L67

${{ env.GOPATH }}/src/github.com/kubeflow/trainer/artifacts/notebooks/*

We might also want to update the default output path:
https://github.com/kubeflow/trainer/blob/master/Makefile#L152

[tenzen-y]

Great addition! Let's also modify the path for output Notebooks to not upload manifests to the GitHub: https://github.com/kubeflow/trainer/blob/master/.github/workflows/test-e2e.yaml#L67

${{ env.GOPATH }}/src/github.com/kubeflow/trainer/artifacts/notebooks/*

We might also want to update the default output path: https://github.com/kubeflow/trainer/blob/master/Makefile#L152

I thought uploading manifests would be useful. When we debugging CI, I would like to confirm generated manifests.
However, I agree with creating notebooks subdirectory.

[tenzen-y]

@andreyvelich I created the subdirectory artifacts/notebooks, but I kept uploading the entire artifacts directory.
Do you still think we should not upload the entire artifacts directory?

[andreyvelich]

@andreyvelich I created the subdirectory artifacts/notebooks, but I kept uploading the entire artifacts directory. Do you still think we should not upload the entire artifacts directory?

I think, it is fine to upload the entire artifact directory to GitHub, since in the future we can introduce more tests which might require various configuration of controller manager (e.g. gang-scheduling, leader election, etc.)

[tenzen-y]

@andreyvelich I created the subdirectory artifacts/notebooks, but I kept uploading the entire artifacts directory. Do you still think we should not upload the entire artifacts directory?

I think, it is fine to upload the entire artifact directory to GitHub, since in the future we can introduce more tests which might require various configuration of controller manager (e.g. gang-scheduling, leader election, etc.)

That sounds great

[andreyvelich]

/lgtm
/approve

[google-oss-prow]

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: andreyvelich

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

• OWNERS [andreyvelich]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment