Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[v2compat] umbrella - KFP data model based caching #5667

Closed
7 tasks done
Bobgy opened this issue May 18, 2021 · 6 comments
Closed
7 tasks done

[v2compat] umbrella - KFP data model based caching #5667

Bobgy opened this issue May 18, 2021 · 6 comments
Assignees
Labels

Comments

@Bobgy
Copy link
Contributor Author

Bobgy commented May 18, 2021

/assign @capri-xiyue

@Bobgy Bobgy changed the title [v2compat] KFP data model based caching [Launcher] KFP data model based caching May 18, 2021
@Bobgy Bobgy changed the title [Launcher] KFP data model based caching KFP data model based caching May 18, 2021
@Bobgy Bobgy added the size/XXL label May 24, 2021
@Bobgy
Copy link
Contributor Author

Bobgy commented May 24, 2021

Needs further TODO item break down

@Bobgy Bobgy changed the title KFP data model based caching [v2compat] umbrella - KFP data model based caching May 24, 2021
@juliusvonkohout
Copy link
Member

Hi @Bobgy do you have the following in mind ? #5509
Or will there be a native way such that deleting runs from the pripelines webinterface will also delete the artifacts from minio and make the cacher aware that the files are gone?

@Bobgy
Copy link
Contributor Author

Bobgy commented Jul 5, 2021

@capri-xiyue can introduce more, but our implementation is basically what you described.

When runs are deleted, their entries will no longer be used for caching.

We haven't invented a way to delete external data for users yet, because data loss would be very frightening.

@Bobgy
Copy link
Contributor Author

Bobgy commented Jul 6, 2021

Hi @zijianjoy, I just realized you'll need to make a small change #5977 for caching.
You can coordinate with @capri-xiyue for when and how to do that

@Bobgy
Copy link
Contributor Author

Bobgy commented Aug 20, 2021

All P0s are done, great work @capri-xiyue @zijianjoy

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

No branches or pull requests

3 participants