Skip to content

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Make it easy to get the correct file path of a dataset #3611

Closed
merelcht opened this issue Feb 9, 2024 · 0 comments
Closed

Make it easy to get the correct file path of a dataset #3611

merelcht opened this issue Feb 9, 2024 · 0 comments

Comments

@merelcht
Copy link
Member

merelcht commented Feb 9, 2024

Description

To get the correct (versioned) file path for a dataset is quite hard. There doesn't seem to be one way that works for both non-versioned and versioned datasets and/or local and remote datasets.

The main question here is why users need to access the file path. Not all datasets have a file path, e.g. APIDataset and so it's important to understand the true user need, before diving into solution.

Context

  1. dataset._filepath() for non-versioned, local dataset
  2. dataset_get_load_path() for versioned, local datasets
  3. Remote datasets: get_filepath_str(self._get_load_path(), self._protocol)

This might not even be the full list of ways to get the file path.

Important

This idea is based on observations from several Kedro engineers see e.g. #1778. However, we need a clear view on what user needs are when it comes to why they need the file path and what their use cases are. Any implementation should be preceded by user research: #1978

@merelcht merelcht converted this from a draft issue Feb 9, 2024
@kedro-org kedro-org locked and limited conversation to collaborators Mar 28, 2024
@merelcht merelcht converted this issue into discussion #3753 Mar 28, 2024

This issue was moved to a discussion.

You can continue the conversation there. Go to discussion →

Labels
None yet
Projects
Archived in project
Development

No branches or pull requests

1 participant