Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extend 'preview' mechanism to kedro-datasets #1622

Closed
MehdiNV opened this issue Nov 2, 2023 · 4 comments
Closed

Extend 'preview' mechanism to kedro-datasets #1622

MehdiNV opened this issue Nov 2, 2023 · 4 comments

Comments

@MehdiNV
Copy link
Contributor

MehdiNV commented Nov 2, 2023

Description

Ticket SQL is blocked as existing solutions aren't viable.

The most optimal solution would be modifying kedro-datasets to have a new supporting method for kedro-viz, which would shift responsibility from viz to kedro-plugins instead. Easier to test, less coupling between the two systems.

Context

A change like this is accepted by multiple team members - it's a viable compromise, but still doable

Possible Implementation

Adding the existing 'preview' method in kedro-datasets to ones like SQLQueryDataset.

@MehdiNV MehdiNV added this to Kedro-Viz Nov 2, 2023
@MehdiNV MehdiNV self-assigned this Nov 2, 2023
@MehdiNV MehdiNV moved this to Todo in Kedro-Viz Nov 2, 2023
@MehdiNV MehdiNV moved this from Todo to In Progress in Kedro-Viz Nov 2, 2023
@MehdiNV MehdiNV moved this from In Progress to Todo in Kedro-Viz Nov 2, 2023
@rashidakanchwala rashidakanchwala moved this from Todo to Backlog in Kedro-Viz Nov 13, 2023
@rashidakanchwala
Copy link
Contributor

Refer to the spike work done here - #1623

@astrojuanlu
Copy link
Member

One thing that is not clear to me - is the idea to extend the _describe method to allow this preview mechanism? Given that _describe already returns a dictionary, would be nice to reuse it (as suggested in #1714)

@rashidakanchwala rashidakanchwala moved this from Backlog to Todo in Kedro-Viz Feb 5, 2024
@SajidAlamQB SajidAlamQB moved this from Todo to In Progress in Kedro-Viz Feb 27, 2024
@SajidAlamQB
Copy link
Contributor

I've been trying to understand this issue and have some questions:

Can we confirm the scope of this ticket, is there list of dataset types that need the preview functionality extended?

Will the implementation follow similar structure done in kedro-org/kedro-plugins#504?

@SajidAlamQB SajidAlamQB moved this from In Progress to In Review in Kedro-Viz Mar 18, 2024
@SajidAlamQB SajidAlamQB moved this from In Review to Done in Kedro-Viz Apr 3, 2024
@SajidAlamQB
Copy link
Contributor

Extended preview to the following:

  • ParquetDataset
  • SQLTableDataset
  • pandas.JSONDataset
  • JSONDataset

in kedro-org/kedro-plugins#595

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

4 participants