Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SDK/DSL: Enable the deletion of a resource via ResourceOp method #3213

Merged
merged 2 commits into from
Mar 10, 2020

Conversation

elikatsis
Copy link
Member

@elikatsis elikatsis commented Mar 4, 2020

Related to #1779.

This PR enables performing the delete action on a resource that's highly connected to the workflow (e.g., a PVC created using a VolumeOp).

The approach of #3194 is aligned to this one, but is a subset of it. I decided to submit this PR to have a generic delete() method that would work out of the box with any ResourceOp (or subclass).

cc @PatrickXYS


This change is Reviewable

@PatrickXYS
Copy link
Member

Your approach is generic, but I have some questions from users' side:

  1. How can we make sure that our users who use ResourceOp want to have to separate ResourceOp.delete()? And of course VolumeOp want this method, but we need to gather more information about other people who use ResourceOp. Probably ping more people take a look and confirm the idea. @Ark-kun @IronPan @Jeffwan @gaoning777 @hongye-sun

  2. It's a little bit confusing with having a passing parameter as action and a separate .delete() method, if we have to do it this way, we need to be very careful about writing documentation for our users.

  3. I suggest you mention your change in ResourceOp.__init__() function's comment, and tell our users use .delete() if they want to utilize delete operation.

Overall, it looks great and if we confirm users want to have a separate .delete() let's do it this way.

cc @elikatsis

@elikatsis
Copy link
Member Author

Your approach is generic, but I have some questions from users' side:

  1. How can we make sure that our users who use ResourceOp want to have to separate ResourceOp.delete()? And of course VolumeOp want this method, but we need to gather more information about other people who use ResourceOp. Probably ping more people take a look and confirm the idea. @Ark-kun @IronPan @Jeffwan @gaoning777 @hongye-sun

VolumeOp is a subclass of ResourceOp. [VolumeOp extends ResourceOp. These two phrases have the same meaning].
So a VolumeOp inherits every ResourceOp property, attribute, method, etc., e.g., the method delete().
The class VolumeOp may decide to override a method if the parent class implementation isn't sufficient. For example, if the ResourceOp method delete() isn't sufficient for a VolumeOp, then it can have its own.
In this case, I think there is no need to override it. It seems to me that the tests I've added prove that it's ok, but, of course, if and when you actually use it you may find something that's not functioning properly and we will figure out how to solve it.

  1. It's a little bit confusing with having a passing parameter as action and a separate .delete() method, if we have to do it this way, we need to be very careful about writing documentation for our users.
  2. I suggest you mention your change in ResourceOp.__init__() function's comment, and tell our users use .delete() if they want to utilize delete operation.

Overall, it looks great and if we confirm users want to have a separate .delete() let's do it this way.

ResourceOps [and VolumeOps, VolumeSnapshotOps] are entities that enable the manipulation of resources. Such manipulation may mean create, patch, apply, delete, etc., and that's what the argument action is for.
The delete() method is a new pipeline step, that deletes a resource already manipulated in a previous pipeline step.
So creating a ResourceOp with action="delete" is different from running .delete() on an already defined ResourceOp.

Did I get your arguments correct?

@PatrickXYS
Copy link
Member

@elikatsis Thank you for explanation!

@elikatsis
Copy link
Member Author

/retest

@kim-sardine
Copy link
Contributor

I really need this one. it would be great if I can delete pvc in ExitHandler

@PatrickXYS
Copy link
Member

@kim-sardine It's also the feature that we want. @elikatsis Seems like one of the CI test failed? If you're tied up, perhaps I could help with it.

@elikatsis
Copy link
Member Author

/retest

@elikatsis
Copy link
Member Author

@PatrickXYS I don't understand what the error is.

I tried running retest command but the bot didn't listen to me 😢
I tried it again in case it was stuck, but no luck

@PatrickXYS
Copy link
Member

@elikatsis It's weird. Seems like this PR is detached from Prow system. I'm not sure if there's any workaround we could move forward. But submitting another PR should work.

Perhaps mention our discussion in the new PR would help others gather context.

* Add the method delete() to ResourceOps
* Extend ResourceOp & VolumeOp tests

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>
@elikatsis
Copy link
Member Author

/assign @Ark-kun

@Ark-kun
Copy link
Contributor

Ark-kun commented Mar 10, 2020

/lgtm
/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: Ark-kun

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot merged commit c220059 into kubeflow:master Mar 10, 2020
@elikatsis elikatsis deleted the resourceop-delete branch March 13, 2020 23:26
Jeffwan pushed a commit to Jeffwan/pipelines that referenced this pull request Dec 9, 2020
…eflow#3213)

* SDK/DSL: Enable the deletion of a resource via ResourceOp method

* Add the method delete() to ResourceOps
* Extend ResourceOp & VolumeOp tests

Signed-off-by: Ilias Katsakioris <elikatsis@arrikto.com>

* Fix ValueError not being raised
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants