You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
renku run, renku workflow execute and renku datasets commands should be aware of files from mounted cloud storage being used and track them separately/with added metadata.
We don't really need to track mount commands etc. but we should be aware of where a file came from when it is used in a workflow or dataset. We can treat this similar to how we handle external files, just with a pointer of where the file came from.
Currently mounting is only supported in sessions, but the session mounts don't use the renku cli for mounting (and probably never will), so we would need some temp file where nb-service can let the renku cli know which directory is a cloud mount and what storage id it corresponds to. For ad-hoc configured storage we don't have an id and would need to remember the cloud storage url/config instead.
I'd try to come up with a design for this first, timeboxed, before implementing anything, as it could be rather complex
The text was updated successfully, but these errors were encountered:
renku run
,renku workflow execute
and renku datasets commands should be aware of files from mounted cloud storage being used and track them separately/with added metadata.We don't really need to track mount commands etc. but we should be aware of where a file came from when it is used in a workflow or dataset. We can treat this similar to how we handle external files, just with a pointer of where the file came from.
Currently mounting is only supported in sessions, but the session mounts don't use the renku cli for mounting (and probably never will), so we would need some temp file where nb-service can let the renku cli know which directory is a cloud mount and what storage id it corresponds to. For ad-hoc configured storage we don't have an id and would need to remember the cloud storage url/config instead.
I'd try to come up with a design for this first, timeboxed, before implementing anything, as it could be rather complex
The text was updated successfully, but these errors were encountered: