From 564d7827dabbf1742f62f939a344bcda78ad5745 Mon Sep 17 00:00:00 2001 From: Jen Duong Date: Fri, 21 Jun 2024 10:45:17 +0100 Subject: [PATCH] docs: add docs for initialise session and ADR for mid journey saves (#1272) * docs: add session-initialisation.md and open API spec * add mid journey saves ADR --- docs/adr/0007-mid-journey-save-return.md | 215 ++++++++++++++++++++ docs/runner/session-initialisation-oas.yaml | 143 +++++++++++++ docs/runner/session-initialisation.md | 96 +++++++++ 3 files changed, 454 insertions(+) create mode 100644 docs/adr/0007-mid-journey-save-return.md create mode 100644 docs/runner/session-initialisation-oas.yaml create mode 100644 docs/runner/session-initialisation.md diff --git a/docs/adr/0007-mid-journey-save-return.md b/docs/adr/0007-mid-journey-save-return.md new file mode 100644 index 0000000000..050bed29c8 --- /dev/null +++ b/docs/adr/0007-mid-journey-save-return.md @@ -0,0 +1,215 @@ +# Mid journey save and returns + +- Status: [proposed] +- Deciders: OS maintainers [@jenbutongit](https://github.com/jenbutongit) [@superafroman](https://github.com/superafroman) [@ziggy-cyb](https://github.com/ziggy-cyb) + +- Date: [2024-06-13 when the decision was last updated] + +## Context and Problem Statement + +It is currently possible to inject a full or partial session _into_ the runner, but it is not possible to send session data +externally in the middle of a form. For long forms, or forms which require the user to get a document or find out more +information, they are unable to exit the form without losing all their data. + +## Considered Options + +- Option 1 + - Add a flag, or flags (perhaps on certain pages only?), to the form configuration, e.g. `allowSaveAndExit`, which will render a `Save and exit` button at the bottom of each question page + - The data is then POSTed to a 3rd party application, which will then store the data in their chosen database + - Optionally, add a data format to webhook outputs, which matches the state as stored in redis + - Optionally, add a data format to the `/sessions/{formId}` endpoint, which matches the state, as stored in redis + - The 3rd party application must also handle rehydrating the user's session, and managing how they re-enter the application + - after successful exit, show the user a success screen (customised similarly to the current application complete page), or allow the user to be redirected externally. +- Option 2 + - Add a flag, or flags (perhaps on certain pages only?), to the form configuration, e.g. `allowSaveAndExit`, which will render a `Save and exit` button at the bottom of each question page + - The runner stores this in redis with a longer ttl, or another database (possibly postgres), and sends the user a link so that they can return to their session + - after successful exit, show the user a success screen (customised similarly to the current application complete page), or allow the user to be redirected externally. + +## Decision Outcome + +## Decision Outcome + +Chosen option: "[option 1]", only teams with developers have asked for this feature. This will help us reduce the maintenance burden. +It also allows for flexibility on how users return to their journey (e.g. you may have a "task list" page on your application, which is authenticated). + +## Pros and Cons of the Options + +### [option 1] + +- Good, because it encourages engineers to develop in a microservice architecture. In future, if the runner needs to be replaced, + you will not need to rewrite the session hydration part of the application. +- Good, because teams looking to implement this feature may already have a preferred data store, and possibly an API which can serve this data. + XGovFormBuilder will not lock teams into tech they are not familiar with, or add superfluous tech to their stack +- Good, because the only "required" additional technology XGovFormBuilder requires is currently Redis. + However, Redis is not designed for long term storage. +- Good, because it does not lock teams/users into a specific user journey. In this instance, we would only allow users to return via a URL emailed to them. +- Bad, because it raises the bar to entry, a development team will be required. + +We have also suggested that we add additional data formats to /sessions/{formId} and the webhook output. This is to improve +developer experience and simplify making session rehydration calls. The webhook output format which is required by /sessions/{formId} +is fairly verbose. It includes information like the page title, section, key and answer. This is so that there is reduced +data loss, in the event that forms are changed. Unrecognised keys can still be placed in a generic description field, along with the page title. + +Some teams may only care about the key/answer, and accept the risk or have mitigated it in other ways when components names have changed. +This means that there would be an extra step for them to translate to/from this data format. + +The data format changes can be worked on separately to the save and return feature, but this is a good time to add additional support. + +Below is a comparison of the webhook format, and the state format. + +In redis, the data will be stored like so + +```json5 +{ + progress: [], + checkBeforeYouStart: { + ukPassport: true, + }, + applicantDetails: { + numberOfApplicants: 2, + phoneNumber: "123", + emailAddress: "a@b", + languagesProvided: ["fr", "it"], + contactDate: "2024-12-25T00:00:00.000Z", + }, + applicantOneDetails: { + firstName: "Winston", + lastName: "Smith", + address: { + addressLine1: "1 Street", + town: "London", + postcode: "ec2a4ps", + }, + }, + applicantTwoDetails: { + firstName: "Big", + lastName: "Brother", + address: { + addressLine1: "King Charles Street", + town: "London", + postcode: "SW1A 2AH", + }, + }, +} +``` + +When making calls to /sessions/{formId}, the payload can be shortened slightly from the webhook output, since `question`, `title`, and `type` are not required. + +```json5 +{ + name: "Digital Form Builder - Runner undefined", + metadata: {}, + questions: [ + { + category: "checkBeforeYouStart", + fields: [ + { + key: "ukPassport", + answer: true, + }, + ], + index: 0, + }, + { + category: "applicantDetails", + fields: [ + { + key: "numberOfApplicants", + answer: 2, + }, + ], + index: 0, + }, + { + category: "applicantOneDetails", + fields: [ + { + key: "firstName", + answer: "Winston", + }, + { + key: "lastName", + answer: "Smith", + }, + ], + index: 0, + }, + { + category: "applicantOneDetails", + fields: [ + { + key: "address", + answer: "1 Street, London, ec2a4ps", + }, + ], + index: 0, + }, + { + category: "applicantTwoDetails", + question: "Applicant 2", + fields: [ + { + key: "firstName", + title: "First name", + type: "text", + answer: "big", + }, + { + key: "lastName", + title: "Surname", + type: "text", + answer: "brother", + }, + ], + index: 0, + }, + { + category: "applicantTwoDetails", + question: "Address", + fields: [ + { + key: "address", + title: "Address", + type: "text", + answer: "King Charles Street, London, SW1A 2AH", + }, + ], + index: 0, + }, + { + category: "applicantDetails", + fields: [ + { + answer: ["fr", "it"], + key: "languagesProvided", + }, + ], + index: 0, + question: "Which languages do you speak?", + }, + { + category: "applicantDetails", + fields: [ + { + key: "phoneNumber", + answer: "123", + }, + { + key: "emailAddress", + answer: "a@b", + }, + { + answer: "2024-12-25", + type: "date", + }, + ], + index: 0, + }, + ], +} +``` + +### [option 2] + +- Good, because it simplifies enabling this feature. External applications/microservices do not need to be written. +- Bad, because it may require teams to run additional databases. XGovFormBuilder maintainers will need to write multiple adapters for different types of databases. diff --git a/docs/runner/session-initialisation-oas.yaml b/docs/runner/session-initialisation-oas.yaml new file mode 100644 index 0000000000..8768698b4a --- /dev/null +++ b/docs/runner/session-initialisation-oas.yaml @@ -0,0 +1,143 @@ +openapi: 3.0.0 +info: + title: Runner - initialise session + version: 1.0.0 +servers: + - url: http://localhost:3009 + +components: + schemas: + CallbackOptions: + type: object + properties: + callbackUrl: + description: The URL to send the PUT request to, after the user has completed the form + type: string + redirectPath: + description: Which page to send the user to, when after they've activated the session. Defaults to /{formId}/summary + type: string + message: + description: What to display at the top of the summary page + type: string + customText: + description: What to display on the application complete page. It mirrors the ConfirmationPage["customText"] schema. + type: object + properties: + paymentSkipped: + description: Setting to false disables this string from rendering on the application complete page. + oneOf: + - type: boolean + - type: string + nextSteps: + description: Setting to false disables this string from rendering on the application complete page. + oneOf: + - type: boolean + - type: string + components: + description: Any additional content components to display on the application complete page. It mirrors the ConfirmationPage["components"] schema. + type: array + items: + type: object + properties: + name: + type: string + options: + type: object + type: + type: string + content: + type: string + schema: + type: object + required: + - callbackUrl + + Question: + type: object + properties: + question: + type: string + category: + type: string + fields: + type: array + items: + type: object + properties: + key: + type: string + answer: + oneOf: + - type: string + - type: boolean + - type: number + required: + - key + - answer + + Metadata: + type: object + +paths: + /sessions/{formId}: + post: + summary: Submit a session with options and questions + parameters: + - name: formId + in: path + description: This must match the form JSONs' filename + required: true + schema: + type: string + requestBody: + required: true + content: + application/json: + schema: + type: object + properties: + options: + $ref: "#/components/schemas/CallbackOptions" + questions: + type: array + items: + $ref: "#/components/schemas/Question" + metadata: + $ref: "#/components/schemas/Metadata" + responses: + "200": + description: Success + content: + application/json: + schema: + type: object + properties: + token: + type: string + "404": + description: Form not found + content: + application/json: + schema: + type: object + properties: + message: + type: string + "403": + description: Callback URL not allowed + content: + application/json: + schema: + type: object + properties: + message: + type: string + "400": + description: Both htmlMessage and message were provided + content: + application/json: + schema: + type: object + properties: + message: + type: string diff --git a/docs/runner/session-initialisation.md b/docs/runner/session-initialisation.md new file mode 100644 index 0000000000..877e446440 --- /dev/null +++ b/docs/runner/session-initialisation.md @@ -0,0 +1,96 @@ +# Session initialisation (rehydration) + +Sessions may be inserted into the form runner, and then activated by a user, given that they have the token. + +This is more suitable compared to [query-param-prepopulation](./designer/query-param-prepopulation.md) if you need to +prepopulate a lot of data or you do not want to send the user a URL with personal data in it. + +The general flow is: + +1. POST the user's session to `/session/{formId}` +1. The session will be stored in session storage (usually redis), as `{ [generatedToken]: sessionData }` +1. The POST request will respond with the generatedToken. The user will use this to activate their session +1. Your service sends the user (either by email, or showing them a URL on your service) + to `https://runner-url/session/{generatedToken}` +1. The user will be redirected to the configured pages (or to the summary page by default) +1. The session data is copied from `{ [generatedToken]: sessionData }`, to where it would "usually" go, which + is `{ [formId]: sessionData }` +1. Once activated, the token will be revoked, it cannot be used again + +## Environment variables + +| variable | type | example | description | +| ----------------------------- | -------- | --------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | +| SAFELIST | string[] | your.service.gov.uk | These are the allowed hostnames you wish to PUT data to, after the user has completed the form from a rehydrated session | +| INITIALISED_SESSION_TIMEOUT | number | 2419200000 | Time, in ms, you wish to keep a rehydrated session in the redis instance for. It will delete after this time if a user does not activate it | +| INITIALISED_SESSION_KEY | string | super-s3cure-p4ssw0rd | The user's token is generated with this key, similarly, the user's session is decrypted with this key. ⚠️ You must ensure this is set if you are deploying replicas. You must also ensure you re-issue tokens if you change this key. | +| INITIALISED_SESSION_ALGORITHM | string | HS512 | HS512 is the default. You may use: `RS256`, `RS384`, `RS512`,`PS256`, `PS384`, `PS512`, `ES256`, `ES384`, `ES512`, `EdDSA`, `RS256`, `RS384`, `RS512`, `PS256`, `PS384`, `PS512`, `HS256`, `HS384`, `HS512`. ⚠️ You must reissue your tokens if you change this algorithm. | + +## Initialising a session + +See [session-initialisation-oas.yaml](./session-initialisation-oas.yaml) for the Open API specification. + +Sample payload for POSTing to `/session/{formId}`: + +```json5 +{ + options: { + callbackUrl: "https://b4bf0fcd-1dd3-4650-92fe-d1f83885a447.mock.pstmn.io", + redirectPath: "/summary", // after session is activated, user will be redirected to ${formId}/${redirectPath} + message: "Please fix this thing..", //message to display to the user on the summary page + customText: { + //same as ConfirmationPage["customText"] + paymentSkipped: false, + nextSteps: false, + }, + components: [ + // same as ConfirmationPage["components"] + { + name: "WLskhZ", + options: {}, + type: "Html", + content: "Thanks!", + schema: {}, + }, + ], + }, + questions: [ + { + fields: [ + { + key: "size", + answer: "Large firm (350+ legal professionals)", + }, + ], + }, + { + question: "Can you provide legal services and support to customers in English?", // optional. This makes no difference, but it is what is originally sent on a "fresh" application + category: "mySection", //optional - category is renamed to "section". You MUST provide the category/section if your form uses sections. + fields: [ + { + key: "speakEnglish", + answer: true, + }, + ], + }, + ], + metadata: { id: "abc-001" }, // any additional information you'd like to send to the callback Url +} +``` + +Sample response for POSTing to `/session/{formId}`: + +```json5 +{ + token: "efg-hi5-jk7", +} +``` + +The session will now be available for one time use at `localhost:3009/session/efg-hi5-jk7`. The user must submit the form, +otherwise they will need to request a token from you again. + +You may generate and email tokens on an automated basis, for example, once a year. But it is recommended that you have a +"landing page" on an external application, which presents the user with a link or button. After clicking this link, +your external application should then generate the token, and redirect the user to it. +It means you do not have to worry about tokens expiring, or reissuing tokens if your INITIALISED_SESSION_KEY +or INITIALISE_SESSION_ALGORITHM has changed.