You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Experiment Tracking relies on Tracking Datasets and we create TrackingDatasetModel before we discover and get all the datasets that are defined using dataset factory pattern
Ideal way would be to implement this discovery on the kedro side, but if not possible, we need to discuss on how to discover the datasets. Earlier we used .exists() on the datasets for discovery which caused the below issues -
Users were not able to use Kedro Viz if the dataset defined in the catalog does not exist (This was resolved using exception handlers)
Users received a timed-out issue as the datasets resided remotely and in a distributed cluster
Experiment Tracking relies on Tracking Datasets and we create TrackingDatasetModel before we discover and get all the datasets that are defined using dataset factory pattern
Ideal way would be to implement this discovery on the kedro side, but if not possible, we need to discuss on how to discover the datasets. Earlier we used .exists() on the datasets for discovery which caused the below issues -
Parent issue - #1480
The text was updated successfully, but these errors were encountered: