Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix uninstall + remove left over resources after test failure #3605

Closed
wants to merge 5 commits into from

Conversation

carlisia
Copy link
Contributor

Does your change fix a particular issue?

Fixes #3599

Please indicate you've done the following:

Carlisia added 2 commits March 17, 2021 15:05
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
@carlisia carlisia added kind/changelog-not-required PR does not require a user changelog. Often for docs, website, or build changes kind/release-blocker Must fix issues for the coming release (milestone) labels Mar 20, 2021
@carlisia carlisia changed the title Fix uninstall + left over resources after test failure Fix uninstall + remove left over resources after test failure Mar 20, 2021
Carlisia added 2 commits March 19, 2021 20:48
Signed-off-by: Carlisia <carlisia@vmware.com>
Signed-off-by: Carlisia <carlisia@vmware.com>
Copy link
Member

@ashish-amarnath ashish-amarnath left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@carlisia I know that this PR is in draft state.
Considering that this is blocking the release, trying to reduce the latency here. So I made some comments. PTAL.

}

rolebinding := install.ClusterRoleBinding(veleroNamespace)
time.Sleep(time.Second * 60)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Using magic periods of sleep makes this very brittle.
Please use WaitForNamespaceDeletion where we wait for a known condition.

Suggest calling client.CoreV1().Namespaces().Delete API and then WaitForNamespaceDeletion
This way we need not recreate a namespace object (Line 78)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, I not at all had the intention to leave this here :) - it needs to be user configurable.

Example: `# velero uninstall -n backup`,
Run: func(c *cobra.Command, args []string) {

if !cli.GetConfirmation() {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This simply presents the user with

		fmt.Printf("Are you sure you want to continue (Y/N)? ")

I would suggest printing a message describing the action we are attempting before calling cli.GetConfirmation

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's add a --quiet or --force option so this is scriptable

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good call, thanks.

Comment on lines +99 to +110
crb := install.ClusterRoleBinding(namespace)
key = kbclient.ObjectKey{Name: crb.Name, Namespace: namespace}
if err := kbClient.Get(context.Background(), key, crb); err != nil {
if apierrors.IsNotFound(err) {
fmt.Printf("Velero installation rolebinding %q does not exist, skipping.\n", crb.Name)
} else {
errs = append(errs, errors.WithStack(err))
}
} else {
if err := kbClient.Delete(context.Background(), crb); err != nil {
errs = append(errs, errors.WithStack(err))
}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like we hard code the name of the cluster rolebinding to "velero"
https://github.com/vmware-tanzu/velero/blob/main/pkg/install/resources.go#L109

Why not simplify this to delete using the kubernetes API for cluster rolebinding instead of using the generic Delete API?

return errors.Wrap(err, errorMsg)
}
}

fmt.Printf("Velero is installed and ready to be tested in the %s namespace! ⛵ \n", io.Namespace)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: "Velero is installed ..." is a duplicate message.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't see the duplication. Where is the other one?

@@ -104,23 +119,23 @@ func RunKibishiiTests(client *kubernetes.Clientset, providerName, veleroCLI, vel
return errors.Wrap(err, "Failed to generate data")
}

if err := VeleroBackupNamespace(oneHourTimeout, veleroCLI, veleroNamespace, backupName, kibishiiNamespace, backupLocation, useVolumeSnapshots); err != nil {
VeleroBackupLogs(fiveMinTimeout, veleroCLI, veleroNamespace, backupName)
if err := veleroBackupNamespace(oneHourTimeout, veleroCLI, veleroNamespace, backupName, kibishiiNamespace, backupLocation, useVolumeSnapshots); err != nil {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These were all public so we didn't need to put all of the e2e tests in the same package. Eventually I think we want to split this up.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When we do split these tests into separate packages will be a great time to export what is needed to be exported.

Signed-off-by: Carlisia <carlisia@vmware.com>
@carlisia
Copy link
Contributor Author

Closing this issue in favor of braking this work into 3 separate PRs.

For reference:

PR to fix the uninstall issue: #3618

New issues to track the rest:
#3624 - e2e tests not cleaning up after failures
#3625 - Consolidate API clients for the e2e tests

@carlisia carlisia closed this Mar 24, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
has-e2e-tests kind/changelog-not-required PR does not require a user changelog. Often for docs, website, or build changes kind/release-blocker Must fix issues for the coming release (milestone)
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Bug with the Velero uninstall cmd
3 participants