Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make KedroPipelineModel compatible with a broader range of kedro version #483

Open
Galileo-Galilei opened this issue Nov 1, 2023 · 0 comments
Labels
enhancement New feature or request waiting-for-kedro The implementation of this feature is blocked by a ticket in kedro

Comments

@Galileo-Galilei
Copy link
Owner

Description

KedroPipelineModel stores several kedro objects (Datacatalog, Pipeline, Runner). Since this object is pickled, any slight change in one of the underlying kedro objects (e.g. a private attribute) prevents loading with the model if you don't have the exact kedro version used for training.

Context

This forces users to retrain model almost every time they upgrade their kedro version, which is painful when training is slow.

Possible Implementation

Store the configuration of the object (e.g. convert back DataCatalog config to a dict, and the pipeline to a json file).

@Galileo-Galilei Galileo-Galilei added the enhancement New feature or request label Jan 16, 2024
@Galileo-Galilei Galileo-Galilei added the waiting-for-kedro The implementation of this feature is blocked by a ticket in kedro label Oct 28, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request waiting-for-kedro The implementation of this feature is blocked by a ticket in kedro
Projects
Status: ⛔ Blocked
Development

No branches or pull requests

1 participant