build: release 5.1.0

[tshuli]

New

• feat: Add submission limits for submissions #1097
• feat: Add submission limits for submissions #1268
• feat: add timestamp to form submission end page #1110
• feat: change logging of incorrect captcha response from error to warn #1258
• feat: remove MyInfo error_description support #1323
• feat: remove MyInfo support from SPCP endpoints #1314
• feat: upgrade to beta version of myinfo-gov-client #1311

Improvements

• chore: add newline in husky .gitignore #1272
• chore: change copy & insert secretkeyloss golink #1293
• refactor: convert frontend MyInfo services to TypeScript #1233
• refactor: encapsulate parsed responses (part 1) #1140
• refactor: migrate attachment validator to ts #1181
• refactor: migrate checkbox validator to typescript #1225
• refactor: remove try-catch from submissions pipeline #1315
• refactor: stop replacing spaces in SAML artifact #1325
• refactor/logic: Validate webpack upgrades are ready for React migration #1201
• test: add tests for new MyInfo functionality #1313
• feat(frontend): migrate file handler service to Typescript, update frontend test framework and lint #1243
• feat(frontend): write FieldVerificationService in TypeScript #1259

Bug Fixes

• fix: put back link for email reliability #1324
• fix: remove form.msgSrvcName validation and add human error recovery message #1235

Dependency Updates

• chore(deps-dev): bump @babel/core from 7.12.17 to 7.13.8 #1285
• chore(deps-dev): bump @babel/plugin-transform-runtime #1263
• chore(deps-dev): bump @babel/preset-env from 7.12.17 to 7.13.9 #1264
• chore(deps-dev): bump @opengovsg/mockpass from 2.6.3 to 2.6.4 #1267
• chore(deps-dev): bump @types/convict from 5.2.2 to 6.0.0 #1309
• chore(deps-dev): bump @types/mongodb from 3.6.8 to 3.6.9 #1303
• chore(deps-dev): bump @types/node from 14.14.31 to 14.14.32 #1300
• chore(deps-dev): bump @types/node from 14.14.32 to 14.14.33 #1326
• chore(deps-dev): bump @typescript-eslint/eslint-plugin #1253
• chore(deps-dev): bump @typescript-eslint/eslint-plugin #1278
• chore(deps-dev): bump @typescript-eslint/parser from 4.15.1 to 4.15.2 #1217
• chore(deps-dev): bump @typescript-eslint/parser from 4.15.2 to 4.16.1 #1266
• chore(deps-dev): bump @typescript-eslint/parser from 4.16.1 to 4.17.0 #1328
• chore(deps-dev): bump core-js from 3.9.0 to 3.9.1 #1269
• chore(deps-dev): bump eslint from 7.20.0 to 7.21.0 #1289
• chore(deps-dev): bump eslint-config-prettier from 8.0.0 to 8.1.0 #1287
• chore(deps-dev): bump eslint-plugin-jest from 24.1.5 to 24.1.7 #1304
• chore(deps-dev): bump eslint-plugin-jest from 24.1.5 to 24.1.8 #1308
• chore(deps-dev): bump eslint-plugin-jest from 24.1.8 to 24.1.9 #1319
• chore(deps-dev): bump husky from 4.3.8 to 5.1.3 #1105
• chore(deps-dev): bump mongodb-memory-server-core from 6.9.3 to 6.9.6 #1318
• chore(deps-dev): bump stylelint from 13.11.0 to 13.12.0 #1307
• chore(deps-dev): bump stylelint-config-standard from 20.0.0 to 21.0.0 #1321
• chore(deps-dev): bump stylelint-prettier from 1.1.2 to 1.2.0 #1256
• chore(deps-dev): bump testcafe from 1.11.0 to 1.12.0 #1294
• chore(deps-dev): bump ts-jest from 26.5.1 to 26.5.2 #1261
• chore(deps-dev): bump ts-jest from 26.5.2 to 26.5.3 #1295
• chore(deps-dev): bump ts-node-dev from 1.1.1 to 1.1.6 #1281
• fix(deps): bump @babel/runtime from 7.12.18 to 7.13.9 #1276
• fix(deps): bump @babel/runtime from 7.13.9 to 7.13.10 #1327
• fix(deps): bump @sentry/browser from 6.2.0 to 6.2.1 #1262
• fix(deps): bump @sentry/integrations from 6.2.0 to 6.2.1 #1288
• fix(deps): bump aws-info from 1.1.0 to 1.2.0 #1290
• fix(deps): bump aws-sdk from 2.848.0 to 2.856.0 #1280
• fix(deps): bump aws-sdk from 2.856.0 to 2.858.0 #1302
• fix(deps): bump aws-sdk from 2.858.0 to 2.859.0 #1329
• fix(deps): bump bcrypt from 5.0.0 to 5.0.1 #1274
• fix(deps): bump connect-mongo from 3.2.0 to 4.2.0 #1240
• fix(deps): bump connect-mongo from 4.2.0 to 4.2.2 #1286
• fix(deps): bump has-ansi from 4.0.0 to 4.0.1 #1284
• fix(deps): bump libphonenumber-js from 1.9.11 to 1.9.12 #1301
• fix(deps): bump neverthrow from 4.1.0 to 4.1.1 #1255
• fix(deps): bump nodemailer from 6.4.18 to 6.5.0 #1282
• fix(deps): bump twilio from 3.56.0 to 3.57.0 #1275
• fix(deps): bump whatwg-fetch from 3.6.1 to 3.6.2 #1271

[tshuli]

Tests

@tshuli

Submission Limits

• Create a email mode form, and set the limit to zero. Note: Not possible to set limit to 0
• Verify that the form is not visible.
  
• Set the limit to 1 and submit the form.
• Verify that the submission goes through
• Visit the form again and make sure that the form is not visible any more.
• Set the limit to 2 and open two tabs.
• Attempt to submit the form in both tabs. Only one should work.
• Repeat the above using storage mode

Note:
Admin may be confused by the following UX:

• If the admin tab is open while the submission limit is reached, the Settings tab will still show 'your form is ACTIVE' even after the form is deactivated. Navigating to other tabs does not refresh the state. Admin needs to manually refresh page or exit form admin view and reenter form admin view.
• Similarly, Admin can increase the submission limit in the above situation under settings tab after the limit has been reached, and it would appear that the form is active AND submission limit has not been reached. Admin may be confused why the page is still not loading for public respondent.

[tshuli]

Add timestamp to form submission end page
@tshuli

• Submit a form. Check that timestamp appears in form submitted page.

[tshuli]

Myinfo related

@tshuli

Remove MyInfo error_description support

• Log in to a MyInfo form but click "Cancel" on the consent page. Ensure that you see the MyInfo error when redirected back to the form, and that there is matching message in the logs (message: 'MyInfo returned error from consent flow') which contains an errorDescription.

Remove MyInfo support from SPCP endpoints

• Activate and submit a SingPass form
• Activate and submit a CorpPass form

remove try-catch from submissions pipeline

• Submit SingPass form in admin preview mode
• Submit CorpPass form in admin preview mode
• Submit MyInfo form in admin preview mode

[tshuli]

@mantariksh
encapsulate parsed responses

• Create a corppass form with email autoreply and pdf response. Submit form. Check that the autoreply email and pdf response both have a masked UID number with only the last 4 characters shown (e.g. *****123A)
• Add a text field to the form with the title 'CorpPass Validated UID'. Check that the response on the text field is not masked in autoreply.
• Check that the email response to the admin still has the full UID number.

[tshuli]

@karrui
migrate attachment validator to ts

• Create two required attachment fields with 1mb size limit each. Hide one of them behind logic. Check that normal submission using frontend succeeds for both visible and hidden field.
• Check that following validation works when submitting programatically
• Attachment size cannot be above the specified limit. Use the AngularJS inspector to modify field size limit to 2mb on frontend. Upload a file of >1mb. Check that submission is rejected "File size more than limit"
• Attachment cannot be empty if required. First submit a form with the second attachment field hidden. Programatically resubmit with the second field unhidden but with no answer or attachment. Check that submission is rejected "Answer is empty string"
• Now inject an answer into the response for the unhidden field but with no attachment content. Check that submission is rejected "No attachment content"

[tshuli]

@karrui
migrate checkbox validator to typescript

• Create two required checkbox fields. Hide one of them behind logic. Check that normal submission using frontend succeeds for both visible and hidden field.
• Check that following validation works when submitting programatically
• Answer array cannot be empty for required field
• Answer array cannot have less than the specified minimum number of options
• Answer array cannot have more than the specified maximum number of options
• Answer array cannot have duplicate responses from fieldoption
• Answer array cannot have answers which are not in the fieldoption
• If 'others' is enabled, the others response cannot be empty (i.e. answer in answerArray cannot be 'Others: ')
• Answer array can have two identical 'Others' responses as long as one of them is a valid fieldoption, apart from enabling others

[tshuli]

@mantariksh
write FieldVerificationService in TypeScript

• Create a form with no verified fields. The call to POST /transaction should return an empty object {}
• Create a form with some verified fields. The call to POST /transaction should return an object with expiry and transaction id.
• In a form with email verified fields, attempt to verify email. The OTP should be sent successfully to the email.
  • Provide an invalid OTP. Should show invalid OTP message below the OTP input field.
  • Correctly input OTP. Should successfully be verified.
• In a form with mobile verified fields, attempt to verify mobile. The OTP should be sent successfully to the number.
  • Provide an invalid OTP. Should show invalid OTP message below the OTP input field.
  • Correctly input OTP. Should successfully be verified.
• Should successfully submit form with verified fields as per normal

[tshuli]

@karrui

IE11 intranet tests

general run-through

• Create email mode form with all fields and custom start/end page. Activate and submit form. Check that you can receive responses in your intranet inbox.
• Collaborators can be added in email mode.
• Create storage mode form with all fields and custom start/end page. Activate and submit form.
• Response CSV can be downloaded for storage mode.
• Responses can be viewed in Data tab in storage mode.
• Attachments for individual fields can be downloaded from Data tab in storage mode.
• Attachment ZIP for submission can be downloaded from Data tab in storage mode.
• Submission IDs can be filtered in storage mode.
• Collaborators can be added and their collaborating rights edited in storage mode.

Unable to download both attachment and responses csv at the same time, seems like the double download is spazzing secure surfing

Submission Limits

• Create a email mode form, and set the limit to zero.
• Verify that the form is not visible.
• Set the limit to 1 and submit the form.
• Verify that the submission goes through
• Visit the form again and make sure that the form is not visible any more.
• Set the limit to 2 and open two tabs.
• Attempt to submit the form in both tabs. Only one should work.
• Repeat the above using storage mode

Storage mode shows submission error with no good recovery text unlike email mode.
Storage mode shows default "Please refresh the form and try again" whilst email mode shows form deactivated message