Remote file access (external cloud storage)

[jimmywarting]

I think it would be really cool if there where any possibility to hook into cloud storages somehow...

I'm thinking sftp, WebDAV, smb like access... We could define a hole new protocol from the ground up, maybe even based on CORS policy and http2?

Imagine being able to hook into Dropbox or Google Drive and not having to worry about how some complicated SDK of how they handle file/directory handles, permissions, and walking the file tree structure etc. or writing a own custom REST api for managing files or dealing with file locking

it would be super cool if we could define a web based protocol to communicate with cloud storages with the same set of permission request model with reading/writing/seeking in a "standard" way of doing things like move, delete and copy things. on the server without actually downloading the hole structure and re upload it

I'm pretty sure it could be possible to get File class that isn't represented by some memory data or a filesystem path to some physical place on your own harddrive.
There is already something analogous in the http2 implementation in the form of the respondWithFile() and respondWithFD() APIs in the http2 side. and browser already have some way to represent a File that don't yet have any data.
so maybe it could be possible to get a File object that yet haven't been downloaded?
getting a file with remoteHandle.getFile() would only get metadata of the file such as filename, size and type somehow without actually downloading the content of the file. only when you create some reader/writer would you actually perform some upload/download.

const directoryHandle = await navigator.storage.getRemoteDirectory('https://example.com/fsa/users/003/')
const directoryHandle = await navigator.storage.getRemoteDirectory('/fsa/users/003/')
const directoryHandle = await navigator.storage.getRemoteDirectory('https://webdav.icloud.com')
const directoryHandle = await navigator.storage.getRemoteDirectory('sftp://address:port/path')
const directoryHandle = await navigator.storage.getRemoteDirectory('smb://user:[email protected]/path')

// edit, another suggestion too:
const directoryHandle = await navigator.storage.getRemoteDirectory( await FileSystemHandle.getUniqueId() )

Edit: after reading up on WebDAV a bit more it's essentually already a remote network drive covered by normal http request already and is pretty wildly used by quite many operation systems/application (one such thing is icloud's WebDAV)
i also just discovered that status code 207 is primarily defined for multistatus/webdav stuff and this xml based protocol is pretty annoying to work with, something newer and more modern would have been nicer to work with...

[bradisbell]

Something similar can be done today with Service Workers, but unfortunately implementers have chosen to assume Service Workers are only useful for caching. Therefore, when using various privacy modes or caching disabled, the Service Workers are disabled.

[jimmywarting]

@bradisbell you sparked my interest with "can be done today with Service Workers"

how would you be able to get (say a) FileSystemDirectory handle from using a service worker and something coming from the service worker/network?

do you know any related issues/PR to my proposal and this service worker you talk about that i don't know about?

[josephrocca]

@photopea has expressed a desire for something along these lines: photopea/photopea#2118 (comment)

[photopea]

Since the File System Access API is asynchronous, any service can create an interface (a Javascript library) to provide their files under the File System Access API.

If your app already uses the File System Access API, the only change would be rewriting six letters: window.showDirectoryPicker() to GDRIVE.showDirectoryPicker() , and suddenly, your app would be communicating with Google Drive instead of the local device. The login etc. would be performed by the library, while your app is "await"ing.

So there is nothing W3C needs to do, it is companies like Google, who should create libraries like that ("GDRIVE"), to provide their services under a "standard" interface. Or someone needs to implement a "layer" between the GDrive API and the File System Access API.

I made a so-called "Storage API", which you can see at www.Photopea.com on the left side. Photopea communicates with each storage the same way. There are storages for the file system, Google Drive, Dropbox, and I am planning to add others in the future. I even have a Storage which stores in cookies, but it is a "toy" and is not public :)

[jaime-rivas]

I think that a standardized API to access remote files directly from the browser would be a wonderful addition to the web platform and it is probably one of the key open issues to remove dependency from the underlying operating system.

I have opened an issue in the Chromium's issue tracker so hopefully we can all push forward this new API.

Feature: Remote File Access with standard API to Cloud Storage providers

[josephrocca]

@photopea (and others here), you might like to comment on this related proposal by Google:

• Add cloud file handling proposal #411
• https://github.com/alex292/file-system-access/blob/cloud-file-handle/proposals/CloudFileHandle.md

[jaime-rivas]

Thanks @josephrocca for the heads up!

[photopea]

I think most of cloud services are able to communicate over the HTTP / HTTPS protocol. There already exist standards for web authentication / authorization.

So I think a web app can communicate with a cloud directly, there is no need to do it through a specific browser interface (in that case, the user would have to tell the browser the list of all their cloud storages).

It would be nice if Dropbox, GoogleDrive, OneDrive and others agreed on the same HTTP interface, but can not see any need to do it through a special browser interface.