You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
To get the correct (versioned) file path for a dataset is quite hard. There doesn't seem to be one way that works for both non-versioned and versioned datasets and/or local and remote datasets.
The main question here is why users need to access the file path. Not all datasets have a file path, e.g. APIDataset and so it's important to understand the true user need, before diving into solution.
Context
dataset._filepath() for non-versioned, local dataset
dataset_get_load_path() for versioned, local datasets
This might not even be the full list of ways to get the file path.
Important
This idea is based on observations from several Kedro engineers see e.g. #1778. However, we need a clear view on what user needs are when it comes to why they need the file path and what their use cases are. Any implementation should be preceded by user research: #1978
This discussion was converted from issue #3611 on March 28, 2024 11:30.
Heading
Bold
Italic
Quote
Code
Link
Numbered list
Unordered list
Task list
Attach files
Mention
Reference
Menu
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
Description
To get the correct (versioned) file path for a dataset is quite hard. There doesn't seem to be one way that works for both non-versioned and versioned datasets and/or local and remote datasets.
The main question here is why users need to access the file path. Not all datasets have a file path, e.g.
APIDataset
and so it's important to understand the true user need, before diving into solution.Context
dataset._filepath()
for non-versioned, local datasetdataset_get_load_path()
for versioned, local datasetsget_filepath_str(self._get_load_path(), self._protocol)
This might not even be the full list of ways to get the file path.
Important
This idea is based on observations from several Kedro engineers see e.g. #1778. However, we need a clear view on what user needs are when it comes to why they need the file path and what their use cases are. Any implementation should be preceded by user research: #1978
Beta Was this translation helpful? Give feedback.
All reactions