Add WebServer as a device

[xelahalo]

A little background information:
We are trying to add a feature in @cph-cachet/carp-webservcies-spring, which would allow for a study to be hosted in an anonymous mode, meaning that participants would only receive a link in their inbox, and they wouldn't have to bother with the onboarding process.

When we are generating the links with the help of our authorization service, we need to have a url that we can redirect users. At the time of creating the accounts and links we don't have any deployed groups and devices so I can really only rely on the protocol to contain some sort of information about where the study is hosted.

Do you think this is the right way to approach this @Whathecode? Or do you have any other ideas? Right now we would only be using anonymous invitations for ICAT, but in the future there may be other studies where we want to avoid using accounts and I also don't want to hardcode it on the CAWS level.

[Whathecode]

I can dive a bit deeper later, but wanted to give some quick initial feedback already now.

I think two requirements are confounded here:

1. The ability to invite yourself to a study, or in other words, "open studies". In this case, there is no need to send participants a personalized link. You can direct them to a single URL, and the web server can upon request of the participant create a new deployment and proceed from there.
2. The ability to represent web pages which are supposed to be opened by the participant (as opposed to are opened by a study runtime, which we support through WebTask) as part of tasks in a study protocol.

We considered these use case during early design, and I'm fairly certain it was on a backlog somewhere. I suspect an iMotions backlog which also contained larger overarching design discussions (crossing core/front-end/back-end).

@ltj can maybe help refresh my memory.

For (1), I think we concluded this is mostly an infrastructure concern, if not, my suspicion would be this should be managed somewhere in the studies subsystem.

For (2), I think we had a similar idea of exposing this as a "web server", or "web service". It makes sense; the "study runtime" in this case is a web server. It will have to know which deployment it is collecting data for, etc., though, so it will have to act as a full "client".

P.s. I think you'll also have to add a JSON schema file. I thought I added tests that fail if you forgot to do that. 🤔

[xelahalo]

1. Yeah I remember that this was the conclusion of our previous conversation, but for example in the case of ICAT it is not technically an "open" study, but instead a study where only a set number of participants should have access to it. This is like standard invitations, however, we don't really have access to their e-mail addresses, but instead the researchers would want to send these links to the participants' e-boks or digitalmail or some state maintained inbox in general. We don't really have the integration for e-boks, but even if we had that we would still want to avoid the onboarding process, and for that I somehow have to figure out where to redirect the users as it is not necessarily the case that the same server will initiate the invitation process. To work around this we have few options:

• when generating we take the url as parameter to the endpoint
• we can lookup what the url is based on some study config ( when I looked at protocol examples, a new device seemed like a reasonable place to put this into )
• we can hack and hardcode the url for the specific study

I kind of want to avoid the last one. We can do the first one as a workaround for now, but it feels a little clunky.

2. Okay 😃 ( I don't really get the "knowing which deployment it collects data for", don't deployments have connected devices? )
   Thanks for the heads up for the schema file. I can also write the tests to check if they exist 😄

[Whathecode]

it is not technically an "open" study, but instead a study where only a set number of participants should have access to it. This is like standard invitations, however, we don't really have access to their e-mail addresses

I probably need to hear more about the exact use case, but this makes me believe this has more to do with an alternative to EmailAccountIdentity.

A user can retrieve study invitations based on their account ID. They get access to that by being authenticated using an account identity they have access to. Email is the predominant one CARP was designed for. But, you can see alternatives were considered in the form of UsernameAccountIdentity.

Maybe this is also relevant/related. 🤔

When we are generating the links with the help of our authorization service, we need to have a url that we can redirect users. 

It sounds like you want this service to generate identities for the users, and then forward that via email/eboks. I.e., the identity is no longer email; it is just submitted via email.

You could generate unique identities (maybe UsernameAccountIdentity is a bit of a misnomer, but it could be modified, or another added) and include that identity in the url you send to the user, so that the webserver can retrieve the matching study deployment.

[Whathecode]

@bardram @xelahalo I created an issue for one part of what I understood is being asked for in this PR, in addition with some background of the original design/what is currently in place. Could you provide answers the the questions asked there?

And as per my first response, I think there are two asks here. The second relates to "browser devices". I definitely think there is something there, but putting the url in the device configuration goes counter CARP's design. The equivalent would be putting a hardcoded BLE address of a specific Bluetooth device to use in a study in the study protocol. That hardly makes it reusable, as it pertains to exactly one deployment. You should consider the protocol to only describe the study; suppose somebody else would want to replicate the study, but host it elsewhere.

The logical place to put a URL would be the DeviceRegistration. (Note that a specific study/backend can pre-register devices. And, device registrations are returned to users as part of retrieving their invitations. This could be another path to pass a URL if needed; but it is unclear to me why, since ICAT should manage this.)

A [DeviceRegistration] uniquely identifies a [DeviceConfiguration] once it is deployed in a study,
contains the necessary details for the client on how to connect to the device, and stores device specifications which may be relevant to the researcher when interpreting collection data,
such as brand/model name and operating system version.

Putting things together, I thus think the real ask here is to:

1. The ICAT application needs to be able to generate a unique URL which the researcher can use to send to a participant, so all the user needs to do is click (username/password can do that, simply by embedding it in URL).
2. When a new URL is requested, ICAT needs to call the authentication service to create a new identity.
3. It should be possible to embed this identity/authentication in the ICAT URL being generated (EmailIAccountIdentity can't do that, but UsernameAccountIdentity could).
4. The ICAT application creates a deployment for an ICAT protocol and invites the newly created user by this identity (not email).
5. When the URL is visited, ICAT uses the information embedded in the URL to authenticate and get invitations, finds the ICAT invitation, and get the study deployment information (just like an ordinary CARP client). It can call registerDevice (maybe uploading things like browser used if it matters for stimuli presentation, etc.), call deviceDeployed and the study can start.
6. Optionally, the ICAT application provides an option to change the password. Changing the username would be a bigger pain, unfortunately, since then CARP core would need to be able to change identities mapped to accounts, which it doesn't do yet (but probably should).

If the goal is to make a more generic solution in CAWS, a CAWS front-end could create participant URLs in the same vain. When the user "logs in" using that URL, an overview of studies they are part of could be shown (or skipped if there is just one). For "website studies" (studies using a Browser device configuration), it could extract the DeviceRegistration which holds a url (as configured in CAWS as part of study setup), which redirects the username/password in a known format to a ICAT url.

[Whathecode]

@xelahalo Regarding JSON schemas, it seems like it is at least documented in the checklist on how to add new device configurations.

Update JSON schemas for the new type:
Add a new schema in rpc/schemas/common/devices corresponding to the class name (e.g., BLEHeartRateDevice.json).
Add the new schema as a subtype in DeviceRegistration.json or DeviceConfiguration.json. The existing examples should guide you, but double-check you specified the right type constant.
Warning: the presence or validity of this schema is not yet tested. It is recommended to serialize an instance of the new data type and validate the output manually for now.

So #404 would be the task to pick up if you want to write that test. ;)

[bardram]

And as per my first response, I think there are two asks here. The second relates to "browser devices". I definitely think there is something there, but putting the url in the device configuration goes counter CARP's design. The equivalent would be putting a hardcoded BLE address of a specific Bluetooth device to use in a study in the study protocol. That hardly makes it reusable, as it pertains to exactly one deployment. You should consider the protocol to only describe the study; suppose somebody else would want to replicate the study, but host it elsewhere.

This is correct.

However, you write:

Putting things together, I thus think the real ask here is to:

1. The ICAT application needs to be able to generate a unique URL that the researcher can use to send to a participant, so all the user needs to do is click (username/password can do that, simply by embedding it in URL).

I'm not sure this is what we want @xelahalo ? Generating anonymous (or "magic") links for participants applies to ALL studies (which are marked as "anonymous"), also non-ICAT ones. This is a feature we also plan to use in our mobile sensing studies - the user would simply get a QR code and can start uploading data w/o the need for the awful login procedure (the largest usability problem we have in all our studies is that users (think: very old woman) can't log in).

So - CAWS should be able to generate such anonymous accounts at "its core".

If the goal is to make a more generic solution in CAWS, a CAWS front-end could create participant URLs in the same vein.

This is exactly the goal and we have already designed this "front-end" where researchers can generate a number of anonymous accounts.

[xelahalo]

And as per my first response, I think there are two asks here. The second relates to "browser devices". I definitely think there is something there, but putting the url in the device configuration goes counter CARP's design. The equivalent would be putting a hardcoded BLE address of a specific Bluetooth device to use in a study in the study protocol. That hardly makes it reusable, as it pertains to exactly one deployment. You should consider the protocol to only describe the study; suppose somebody else would want to replicate the study, but host it elsewhere.

@Whathecode I see this point, but then what about WebTasks? They are also serialized into the protocol, and they also contain a url.

The logical place to put a URL would be the DeviceRegistration. (Note that a specific study/backend can pre-register devices. And, device registrations are returned to users as part of retrieving their invitations. This could be another path to pass a URL if needed; but it is unclear to me why, since ICAT should manage this.)

In the first mental draft of this pr I also considered this, but then I would only be able to access what I need once I actually have a studyDeployment. For that I need

2. When a new URL is requested, ICAT needs to call the authentication service to create a new identity.

before I create the link, so that I have an identity that I can turn into a participant. But if I do that and let ICAT generate the url, then it is technically an open study as anyone would be able to get a link. If we, however, let a client with appropriate access control, like the portal, request the link generation, then what we basically have is my first point:

• when generating we take the url as parameter to the endpoint

Since we still need to know which client/study to initiate the generation for. That means that I don't actually need a new device, or a new device registration.

Okay so I think my conclusion is that for now we will:

1. take the url as input for generating the link (we don't want ICAT to handle this, I think that would introduce unnecessary complexities @bardram)
2. deal with Allow study subsystem to retrieve input elements for participant attributes #463 and Add participants to studies identified through other mechanisms than email #459
3. not create a new DeviceConfiguration or DeviceRegistration
4. take a look at Add test to ensure valid JSON schema is available for all domain objects #404 later, as I have a lot of tasks piled up already

@Whathecode as always, thanks for the input, much appreciated 😄

[Whathecode]

@Whathecode I see this point, but then what about WebTasks? They are also serialized into the protocol, and they also contain a url.

From the WebTask documentation:

Redirects to a web page which contains the task which needs to be performed.
... // The execution of the task is delegated to a web page, so this task uploads no data.

There is thus very limited integration with CARP's domain model, and the full description of the task is the URL. It was introduced to integrate with external survey websites, like Google Forms. That's also why it expects to pass participant, deployment, and trigger ID to the URL. So that when you extract data from the external website, you can manually link things back together.

In contrast, ICAT (at least the impression I have) is intended to become a CARP client. We can move towards modeling the subtasks of ICAT as tasks, or at a minimum have a ICATTask with a typed data point to be uploaded on completion. The linking would happen out of the box.

If you don't do this, there is really no point of integrating ICAT in CARP.

tldr; WebTask suffers from similar problems we are exactly trying to solve by using the CARP domain model. :) For a survey, you should instead also be using a SurveyTask. That simply doesn't exist yet.

[Whathecode]

Since we still need to know which client/study to initiate the generation for. That means that I don't actually need a new device, or a new device registration.

Not certain I follow. It is definitely clear, however, you expect the CAWS portal to manage ICAT studies (context I was missing). That means that the CAWS portal needs to set up the URL indeed, instead of ICAT. My suggestion for the generic solution was you don't create study-specific URLs. You create CAWS URLs for participants, and when visiting the URL (hosted by CAWS webserver) you either present a study overview (theoretically, the participant could be part of multiple studies), or simply redirect to ICAT (if/assuming there is only one study).

take the url as input for generating the link (we don't want ICAT to handle this, I think that would introduce unnecessary complexities @bardram)

This in contrast, would couple participant creation to a study. That seems undesirable to me. Furthermore, you can't pass all the information ICAT needs to upload data for that participant, like the deploymentId! The only thing you can pass is the account identity, which granted, the ICAT implementation could use to retrieve the deployment, but conceptually it feels out of scope and is logic which could be encapsulated by CAWS.

You still need to configure the URL somewhere, though, and this will need to be done through CAWS then. My suggestion was to use the existing CARP endpoints for this. You can create a Website device with a WebsiteRegistration, and pre-register this to capture the ICAT url (with the actual parameters for that specific participant) once the participants are invited. All of this is application-specific code for CAWS to deal with "website studies", and requires the addition of a new Website device to CARP core. This will work for other studies as ICAT as well, but you will need to align on some common mechanism to inject auth/deployment details to the configured URL.

And, ICAT will need to be CARP-aware (unless you want an implementation like WebTask which as I argue before, has little benefit). Concretely, ICAT needs to be able to receive the required authentication mechanism, and either assume there is only one deployment for that account, or, also receive a deploymentId to operate on. How to pass that is an infrastructure concern.

[Whathecode]

Replaced by: #465