Revisit the unmanaged vfolder with the latest storage-proxy architecture

[achimnol]

Unmanaged vfolders allow superadmins to map a random host path to a vfolder, in case that users need to cooperate with external systems which use a specific host path or sub-path in shared network volumes, not following our quota-scope and vfolder path mangling scheme.

Note

Characteristics of unmanaged vfolders:

• It assumes that all agents have the same host-side paths that map to unmanaged vfolders.
• Users cannot create or delete unmanaged vfolders in Backend.AI, while they can mount them into the sessions.
• Unmanaged vfolders are not accelerated in terms of the filesystem operations as we don't have information about its storage host but only the agent path. Depending on the agent's OS and storage configuration, GDS may work though.
  • They don't have the owner vfolder host.
• Unmanaged vfolders cannot be cloned.
• Only admins can create unmanaged vfolders in the control panel or CLI.
  • Backend.AI does neither perform any filesystem-level validation nor directory creation when creating them. It just adds a database record to keep track of the new unmanaged vfolder.
• They are not part of our quota scopes and the quota management should be configured manually by customer admins.
• When an unmanaged vfolder is deleted, it skips the trash bin and immediately disappears from Backend.AI. Though, since it is unmanaged, actual storage-level deletion should be done by customer admins.

For instance, a vfolder created with ./backend.ai vfolder create --unmanaged unfolder1 /home/joongi/unfolder1,

Issue: unmanaged vfolders do not work with the current backend-aware storage-proxy implementation with explicit vfolder host configurations, as the vfolder-host information is missing for them.

Let's update the implementation as follows:

• If the host path of an unmanaged vfolder contains a known vfolder host's mount path, let's follow the configuration of the matched vfolder host.
• Otherwise, let's introduce a concept of "ghost vfolder host", which provides a default set of vfolder-host permission configurations.