Revamp Application Registry

[ishank011]

Currently, the registry works by simply redirecting requests corresponding to the mimetype to the endpoint specified by a given set of rules.

For example, let's say we want to redirect office format requests to a reva appprovider running at localhost:16001 and codimd ones to one running at localhost:16002. We'll add the following rules to appregistry

[grpc.services.appregistry.static.rules]
"application/vnd.oasis.opendocument.text" = "localhost:16001"
"application/vnd.oasis.opendocument.spreadsheet" = "localhost:16001"
"application/vnd.oasis.opendocument.presentation" = "localhost:16001"
"application/compressed-markdown" = "localhost:16002"

Thus when we receive a request to open a file with the extension .docx in the app provider, the storage provider will resolve the mimetype to be an office text format and this request will be forwarded to the reva at localhost:16001.

@glpatcern can you elaborate what is the desired mechanism?

[glpatcern]

I'll sketch here what I think we need to achieve and then we discuss/evolve from here.

The current full picture is that on top of the above, a site manager would need to configure the following in the WOPI server config:

# URL of your Microsoft Office Online service
oosurl = https://your-oos-server.org
# URL of your Collabora Online service
codeurl = https://your-collabora-server.org:9980

Now what we need is:

1. A mechanism to register a new application with the attributes
   {appName, appURL, [list of supported mimetypes/file extensions]}.
   Example:
   {"appName" : "Collabora Online", "appURL" : "https://your-collabora-server.org:9980", "mimetypes" : [".odt", ".ods", ".odp", ".rtf", ".docx", ...]}
   This registration step shall take place via API (on top of through config), because eventually this is to be exposed on the site admin's web UI.
   To be noted that the same mimetype/file extension may be handled by multiple applications. I used the file extensions for the example here but we may well stick to mimetypes.

2. A mechanism to store the default app for each supported mimetype: this is also for the site admin in the first place, to set a default for everyone. The same mechanism shall be exposed to users, so that the settings can be stored on a per-user basis.

3. A way to query the app registry for all appNames that support a given mimetype, which should return a list like:
   [{"name" : "Collabora Online", "default" : 1}, {"name" : "Microsoft Office Online", "default" : 0}]
   This is to be used by the UI to populate the "Open with..." menu item for the user.
   I added a filter for this in the AppRegistry CS3 API, but we probably need to extend it.

4. Whenever the user chooses the "default" app, its URL is worked out by the AppProvider from the registry. In all cases, the WOPI server will be agnostic to the app URLs and we can move out that part and the related config.

We should also evolve the AppProvider so it supports non-WOPI apps with some abstraction layer as discussed long ago, but this is another task of course. It might help though to think about e.g. the Root viewer as a simple non-WOPI app for the case of configuring it entirely within Reva, without relying on the WOPI server to provide the app's URL.

[glpatcern]

Further thoughts on this topic:

• On one side, we want the AppRegistry config to entirely live within Reva. This implies a configuration like:

[grpc.services.appregistry.drivers.static.providers]
"localhost:19000" = {"address" = "https://the-collabora-server.org:9980", "name" = "Collabora Online",
    mimetypes = ["application/vnd.oasis.opendocument.text", "application/vnd.oasis.opendocument.spreadsheet",
        "application/vnd.oasis.opendocument.presentation",
        "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
        "application/msword", ...]}
"localhost:19000" = {"address" = "https://the-msoo-server.com", "name" = "Microsoft Office Online",
    mimetypes = ["application/vnd.openxmlformats-officedocument.wordprocessingml.document", ...]}
"localhost:19000" = {"address" = "https://the-wopibridge.org", "name" = "CodiMD",
    mimetypes = ["text/plain", "text/markdown", "application/compressed-markdown"]}
...

where the AppProvider at localhost:19000 points to the Reva WOPI driver, and all the end application servers are specified.

• On the other side, the current list of "application endpoints" provided by WOPI (see e.g. https://sciencemesh.cernbox.cern.ch/wopi/cbox/endpoints) is much richer as it contains another dimension, which is very WOPI-specific. Further, that list is obtained via the "WOPI discovery" process, see e.g. https://collabora.cern.ch:9980/hosting/discovery for CERN. This part clearly stays with the WOPI server, which would work it out from the top-level URL as configured in each address values above.

For the latter, I'm thinking to introduce an endpoint in the wopiserver to be used by the WOPI AppProvider in Reva, essentially to say: "this is an app endpoint, go discover its capabilities" -> the wopiserver will GET /hosting/discovery (as it is the case today) and populate its internal dictionary, used afterwards by the protocol. How does that sound? The whole point is to have the wopiserver agnostic about the apps.

[ishank011]

So there's a misunderstanding here. The address field doesn't point to the address of the app (office, codimd, etc.). It stays the same as before, pointing to the app provider service (https://github.com/cs3org/reva/blob/master/examples/ocmd/ocmd-server-1.toml#L88-L94). Also, in reva, an app provider refers to the drivers we understand (at the moment, wopi and demo), and not the actual app (where one driver can handle multiple such apps) .

The config would look like

[grpc.services.appregistry.drivers.static.providers]
"localhost:19000" = {"address" = "localhost:19000", "name" = "Collabora Online",
    mimetypes = ["application/vnd.oasis.opendocument.text", "application/vnd.oasis.opendocument.spreadsheet",
        "application/vnd.oasis.opendocument.presentation",
        "application/vnd.openxmlformats-officedocument.wordprocessingml.document",
        "application/msword", ...]}
"localhost:19001" = {"address" = "localhost:19001", "name" = "Microsoft Office Online",
    mimetypes = ["application/vnd.openxmlformats-officedocument.wordprocessingml.document", ...]}
"localhost:19002" = {"address" = "localhost:19002", "name" = "CodiMD",
    mimetypes = ["text/plain", "text/markdown", "application/compressed-markdown"]}

The workflow would look like:

1. OpenInAppProviderRequest{myfile.docx, read, "Microsoft Office Online"}
2. OpenInAppProvider calls GetAppProviders which returns [{"localhost:19000", "Collabora Online"}, {"localhost:19001", "Microsoft Office Online"}]
3. OpenInAppProvider forwards the request to the wopi driver running at localhost:19001 as it matches the app name.

We need to add a config parameter in the wopi driver to point to which app it should open the file with and this would be handled by the wopi driver code + wopiserver. All the wopi drivers (localhost:19000, localhost:19001, localhost:19002) can point to the same wopiserver but this config parameter would be different for those.

[glpatcern]

For the record, this is where we are after further discussions:

1. In terms of services, the AppProvider will also expose an HTTP service on top of GRPC, so that it can be called from web. This will essentially cover the scope of the current WOPI-OCIS "app".

2. In terms of functionality, we can delegate the "discovery" of which mimetypes a given application is able to handle to the AppProvider driver: the WOPI driver can do that by calling /hosting/discovery on a given appurl, other drivers would have to do it according to the app they support (say e.g. the SWAN driver can simply return .ipynb). This action could well be a specific

3. In terms of AppRegistry configuration, we retain the idea of multiple instances of the AppProvider for each "real" external app, even though some of them are served through the (single) WOPI server. So it would look like:

[grpc.services.appregistry.drivers.wopi.providers]
"localhost:19000" = {"address" = "localhost:19000", "name" = "Collabora Online",
     "appurl" = "https://collabora-online.cern.ch:9980"}
"localhost:19001" = {"address" = "localhost:19001", "name" = "Microsoft Office Online",
    "appurl" = "https://oos.web.cern.ch"}
"localhost:19002" = {"address" = "localhost:19002", "name" = "CodiMD",
    "appurl" = "https://cernbox.cern.ch/byoa/codimd" "appinturl" = "https://codimd.cern.ch:3000"}
 
[grpc.services.appregistry.drivers.swan.providers]
"localhost:19003" = {"address" = "localhost:19003", "name" = "SWAN", "appurl" = "https://swan.cern.ch"}

Note the appurl and appinturl, which are part of the Reva configuration (and are to be provided by the site admin when installing the app)

4. The WOPI server does not need to be pre-configured with the same appurls, but will need to access them as it is required to know the full URL of the final app. So I guess we need to make sure the discovery step is exposed both ways:
   a) As a WOPI /wopi/iop/discovery endpoint for Reva to declare a new app and get back the list of supported mimetypes/file extensions.
   b) As a Reva HTTP endpoint for WOPI to ask back "what about this app?" and get its appurl. Let's not forget that the WOPI server is stateless and may be restarted/load-balanced etc., so the info in a) can be cached but we need b) to restore the cache.

[glpatcern]

One more update following discussions with @diocas:

Instead of point 4. we can simply make sure that the WOPI driver of the AppProvider passes the "decorated" [*] appurls to WOPI on the open call, such that it can use them in the WOPI protocol exchanges with the application server: there's a stat call (CheckFileInfo) that requires those URLs to be given back. In this case, no state is needed in WOPI nor any bootstrap/cache refresh logic. On the contrary the full discovery logic must be part of the WOPI AppProvider.

Going in this direction, I'd create a separate endpoint in WOPI, /wopi/iop/openinapp, where those URLs are mandatory parameters and the new simplified logic is implemented, whereas the current /wopi/iop/open stays there for some time for backwards compatibility, and will eventually be deprecated/dropped.

[*] For a given WOPI application, two "decorated" URLs exist, that point to the viewer UI and the editor UI for a given file type. These can be obtained by calling /hosting/discovery on the WOPI application server.

[labkode]

[glpatcern]

For the registration, the alternate model referenced in my last commit means that we have the App Driver doing the discovery on its own, without involving the WOPI server. It will fetch the list of supported mimetypes/file extensions, as well as the WOPI editurl and viewurl for each type of the given end-user app. Those editurl and viewurl will be passed as extra (required) parameter to WOPI on a new HTTP endpoint, /wopi/iop/openinapp.