[Spike] Create a Proof of Concept for Type Hints for dataset previews

[SajidAlamQB]

Description

See here: #1847

By using a type dict or other typing tools, see if the IDE will be able to provide hints that lets users know about the required structure of the preview dictionary (e.g., index, columns, and data keys for TablePreview). Assess how effective the PoC is in helping users keep to the expected preview format and reducing the likelihood of errors due to incorrect return types or structures.

Checklist

• Include labels so that we can categorise your feature request

[SajidAlamQB]

Hey @astrojuanlu @rashidakanchwala,

I replaced the existing NewType definitions for TablePreview, ImagePreview, PlotlyPreview, and JSONPreview with TypedDict and TypeAlias. Specifically, TypedDict was used for TablePreview to explicitly give its expected structure (e.g., index, columns, and data).

See: kedro-org/kedro-plugins#831

Results:

• Type Hinting in IDEs: I verified that the IDE successfully suggests the required keys (like index, columns, and data) when implementing the TablePreview return value within a preview() method. This would definitely help users in reducing potential errors when creating custom dataset previews.

Issue:

• One issue I found was that TypeAlias is only fully supported from Python 3.10 onwards.
• Kedro-plugins currently supports Python 3.9, we may need look at alternatives or extra steps for backward compatibility, maybe maintaining the NewType approach or using conditional imports depending on the Python version?

Let me know your thoughts.

Example type hint for ImagePreview:

Auto suggestion for TablePreview return:

[rashidakanchwala]

This is amazing! Thanks for the spike.

I had one question. In Juanlu's example he returned a PolarsTable which had a differnt format. He didn't type it out just returned df (which was columns:...) -- does it highlight then too, if this is wrong return type

[SajidAlamQB]

Good point I just tested it out and unfortunately it doesn't highlight it, TypedDict seems to provide basic hints, it doesn't enforce the structure strictly.

[astrojuanlu]

Great job @SajidAlamQB and good call @rashidakanchwala ! Sad that my original issue wouldn't have been caught by this.

It's still a small improvement in DX, so maybe it would be nice to have.

[astrojuanlu]

One issue I found was that TypeAlias is only fully supported from Python 3.10 onwards.

And the small improvement in DX is probably not worth the effort of doing this in a way that it's compatible with Python 3.9...

[SajidAlamQB]

Thanks for the feedback. Since the type hinting provides some improvement in developer experience but doesn't strictly enforce the structure and has compatibility issues with Python 3.9, we'll close this spike. For now, we'll keep the current approach and revisit in the future.