Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend 'preview' mechanism to kedro-datasets #1622

Closed
MehdiNV opened this issue Nov 2, 2023 · 4 comments
Closed

Extend 'preview' mechanism to kedro-datasets #1622

MehdiNV opened this issue Nov 2, 2023 · 4 comments

Comments

@MehdiNV
Copy link
Contributor

MehdiNV commented Nov 2, 2023

Description

Ticket SQL is blocked as existing solutions aren't viable.

The most optimal solution would be modifying kedro-datasets to have a new supporting method for kedro-viz, which would shift responsibility from viz to kedro-plugins instead. Easier to test, less coupling between the two systems.

Context

A change like this is accepted by multiple team members - it's a viable compromise, but still doable

Possible Implementation

Adding the existing 'preview' method in kedro-datasets to ones like SQLQueryDataset.

@rashidakanchwala
Copy link
Contributor

Refer to the spike work done here - #1623

@astrojuanlu
Copy link
Member

One thing that is not clear to me - is the idea to extend the _describe method to allow this preview mechanism? Given that _describe already returns a dictionary, would be nice to reuse it (as suggested in #1714)

@SajidAlamQB
Copy link
Contributor

I've been trying to understand this issue and have some questions:

Can we confirm the scope of this ticket, is there list of dataset types that need the preview functionality extended?

Will the implementation follow similar structure done in kedro-org/kedro-plugins#504?

@SajidAlamQB
Copy link
Contributor

Extended preview to the following:

  • ParquetDataset
  • SQLTableDataset
  • pandas.JSONDataset
  • JSONDataset

in kedro-org/kedro-plugins#595

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

4 participants