Integrate with SDS endpoint to support loading Prepop data

[berroar]

What is the context of this PR?

Adds the ability to integrate with the future SDS endpoint in order to be able to load Pre-populated Questionnaire data into runner.

Adds validation of the Prepop data returned by the SDS endpoint, based on the definitions outlined here: ONSdigital/sds-schema-definitions#1

As this PR only covers the initial integration, the response payload from the SDS endpoint is being validated and returned but not being used.

How to review

Set up a mock SDS endpoint using the new dev script added: python -m scripts.mock_sds_endpoint

Ensure the new env var SDS_API_BASE_URL is set with http://localhost:5003/v1/unit_data.

Use the add-sds-dataset-id branch of launcher which allows you to launch with the sds_dataset_id field that is required in order to request prepop data, and attempt to open the newly added test_supplementary_data survey.

Using sds_data_set_id: 34a80231-c49a-44d0-91a6-8fe1fb190e64 will return mocked supplementary data with a repeat and c067f6de-6d64-42b1-8b02-431a3486c178 will return mocked supplementary data without a repeat.

Check the tests added.

Checklist

• New static content marked up for translation
• Newly defined schema content included in eq-translations repo

[berroar]

Integrate with SDS and validate the response (Mocked API) (3 SP)

[katie-gardner]

Still need to test functionally but some initial comments/questions

[MebinAbraham]

A PR should be raised to merge main into the base branch.

[katie-gardner]

Have now tested functionally and working as I would expect except for survey_id validation

[katie-gardner]

Generally changes looking good 👍
I found it quite hard to functionally test as I missed a few things - now we need to update our .env file with the SDS url and if running the mock sds endpoint script in debug mode, remembering to set the working directory to runner so it can find the files
My bad really but might be useful tips to add to run instructions?

Also are the run instructions anywhere else? Thinking a readme in the doc folder or somewhere else might be good to preserve whats on the PR so people have a reference once its merged

Ah - I think the .development.env and functional tests equivalent need updating for the new env var

[kylelawsonAND]

now we need to update our .env file with the SDS url and if running the mock sds endpoint script in debug mode, remembering to set the working directory to runner so it can find the files My bad really but might be useful tips to add to run instructions?

Also are the run instructions anywhere else? Thinking a readme in the doc folder or somewhere else might be good to preserve whats on the PR so peo

Agree would be good to update the .env files

[berroar]

Generally changes looking good 👍 I found it quite hard to functionally test as I missed a few things - now we need to update our .env file with the SDS url and if running the mock sds endpoint script in debug mode, remembering to set the working directory to runner so it can find the files My bad really but might be useful tips to add to run instructions?

Also are the run instructions anywhere else? Thinking a readme in the doc folder or somewhere else might be good to preserve whats on the PR so people have a reference once its merged

Ah - I think the .development.env and functional tests equivalent need updating for the new env var

Updated now 👍

[Yuyuutsu]

Changes look good to me. Will functionally test once i fix my runner 👍

[katie-gardner]

Two minor non-blocking comments but happy with readme and how its all working, and test updates, nice one

[kylelawsonAND]

Looks great now - nice one 👌

[mgdon650]

Changes look good to me. Runs OK!