This issue was moved to a discussion.
You can continue the conversation there. Go to discussion →
Make it easy to get the correct file path of a dataset #3611
Milestone
You can continue the conversation there. Go to discussion →
Description
To get the correct (versioned) file path for a dataset is quite hard. There doesn't seem to be one way that works for both non-versioned and versioned datasets and/or local and remote datasets.
The main question here is why users need to access the file path. Not all datasets have a file path, e.g.
APIDataset
and so it's important to understand the true user need, before diving into solution.Context
dataset._filepath()
for non-versioned, local datasetdataset_get_load_path()
for versioned, local datasetsget_filepath_str(self._get_load_path(), self._protocol)
This might not even be the full list of ways to get the file path.
Important
This idea is based on observations from several Kedro engineers see e.g. #1778. However, we need a clear view on what user needs are when it comes to why they need the file path and what their use cases are. Any implementation should be preceded by user research: #1978
The text was updated successfully, but these errors were encountered: