Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

build: release v5.13.0 #2099

Merged
merged 38 commits into from
Jun 8, 2021
Merged

build: release v5.13.0 #2099

merged 38 commits into from
Jun 8, 2021

Conversation

karrui
Copy link
Contributor

@karrui karrui commented Jun 8, 2021

Release v5.13.0

New week, new release. This release enables webhook retries for the application. Big stuff. Get excited.

Features

  • feat: enable retries for webhooks #2093

Improvements

  • feat: log form updates #2063
  • feat: sort responses for csv download according to creation date #2028
  • fix: restore typings to some model static methods #2067
  • refactor: replace set hook with validator hook in emailField model #1971
  • refactor(beta): migrate to TypeScript #2058
  • refactor(formApiClientFactory): rearrange types #2061
  • refactor(submissions.client.factory): extract admin form features #1983
  • test(betas): provide coverage 23f9a9f

Dependency updates

  • fix(deps): bump @sentry/browser from 6.5.0 to 6.5.1 #2069
  • fix(deps): bump @sentry/integrations from 6.5.0 to 6.5.1 #2071
  • fix(deps): bump aws-sdk from 2.918.0 to 2.919.0 #2060
  • fix(deps): bump aws-sdk from 2.919.0 to 2.920.0 #2070
  • fix(deps): bump aws-sdk from 2.920.0 to 2.922.0 #2078
  • fix(deps): bump nocache from 2.1.0 to 3.0.0 #2068
  • fix(deps): bump twilio from 3.63.0 to 3.63.1 #2072
  • fix(deps): downgrade typescript to 4.2.4 and pin #2097
  • fix(deps): update mongoose to 5.12.12, update model types #2046

Dev dependencies

  • chore(deps-dev): bump @types/node from 14.17.1 to 14.17.2 #2059
  • chore(deps-dev): bump @typescript-eslint/eslint-plugin #2095
  • chore(deps-dev): bump @typescript-eslint/parser from 4.26.0 to 4.26.1 #2094
  • chore(deps-dev): bump core-js from 3.13.1 to 3.14.0 #2076
  • chore(deps-dev): bump eslint from 7.27.0 to 7.28.0 #2077
  • chore(deps-dev): bump prettier from 2.2.1 to 2.3.1 #2075
  • chore(deps-dev): bump type-fest from 0.20.2 to 1.2.0 #2049

Documentation

  • chore: remove endPage.buttons key from database  #2087
  • docs(readme): remove active contributors #2064
  • docs(script): add scripts to privatize all sp/rp student forms #2073

mantariksh and others added 30 commits June 1, 2021 21:21
chore: merge v5.12.0 into develop
)

* docs(scripts): add scripts to set student logos to selected forms

* doc(script): update script for completeness

* fix: script comments
Bumps [@opengovsg/mockpass](https://github.com/opengovsg/mockpass) from 2.7.2 to 2.7.3.
- [Release notes](https://github.com/opengovsg/mockpass/releases)
- [Commits](opengovsg/mockpass@v2.7.2...v2.7.3)

---
updated-dependencies:
- dependency-name: "@opengovsg/mockpass"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
chore: merge v5.12.1 into develop
* chore(deps-dev): bump type-fest from 0.20.2 to 1.2.0

Bumps [type-fest](https://github.com/sindresorhus/type-fest) from 0.20.2 to 1.2.0.
- [Release notes](https://github.com/sindresorhus/type-fest/releases)
- [Commits](sindresorhus/type-fest@v0.20.2...v1.2.0)

---
updated-dependencies:
- dependency-name: type-fest
  dependency-type: direct:development
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

* fix: remove circular reference from PublicForm type

seems to be introduced by the new Merge implementation in type-fest, go back to using Omit

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kar Rui Lau <[email protected]>
* fix(deps): update mongoose to 5.12.12

* fix: correct types of IEmail/EncryptedSubmissionSchema

* refactor: remove explicit `this` in mongoose models
In preparation to porting betas.client.factory to TypeScript, write a
suite of unit tests to ensure functionality does not change during the
migration. Restructure the AngularJS factory a bit to make code paths
easy to reach.

- Restructure `betas.client.factory` so that its functions are
  accessible both via module import and `angular.module().factory()`
- Allow the functions to take in other beta feature fields for test
  purposes, defaulting to `BETA_FEATURES_FIELD`
- Provide test coverage via jest, crowbar-ing a mock into the angular
  global at runtime, to minimise changes to the module under test
Bumps [@types/node](https://github.com/DefinitelyTyped/DefinitelyTyped/tree/HEAD/types/node) from 14.17.1 to 14.17.2.
- [Release notes](https://github.com/DefinitelyTyped/DefinitelyTyped/releases)
- [Commits](https://github.com/DefinitelyTyped/DefinitelyTyped/commits/HEAD/types/node)

---
updated-dependencies:
- dependency-name: "@types/node"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [aws-sdk](https://github.com/aws/aws-sdk-js) from 2.918.0 to 2.919.0.
- [Release notes](https://github.com/aws/aws-sdk-js/releases)
- [Changelog](https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md)
- [Commits](aws/aws-sdk-js@v2.918.0...v2.919.0)

---
updated-dependencies:
- dependency-name: aws-sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…1971)

* refactor: replace set hook with validator hook in emailField model

* refactor: edit error message and remove repeated pre-validate hook

* fix: allow update if includeFormSummary is false

* refactor: eedit validator and add comments
refactor(beta): migrate to TypeScript
* refactor: shift EditFormFieldParams to types/api

* refactor: shift IPossiblyPrefilledField to types

* refactor: add DuplicateFormBody, FormUpdateParmas and PublicFormViewDto to types/api

* refactor: rreimport dependencies based on previous changes

* fix: remove CreateFormService left over from rebase
Bumps [@sentry/browser](https://github.com/getsentry/sentry-javascript) from 6.5.0 to 6.5.1.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/master/CHANGELOG.md)
- [Commits](getsentry/sentry-javascript@6.5.0...6.5.1)

---
updated-dependencies:
- dependency-name: "@sentry/browser"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [aws-sdk](https://github.com/aws/aws-sdk-js) from 2.919.0 to 2.920.0.
- [Release notes](https://github.com/aws/aws-sdk-js/releases)
- [Changelog](https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md)
- [Commits](aws/aws-sdk-js@v2.919.0...v2.920.0)

---
updated-dependencies:
- dependency-name: aws-sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [twilio](https://github.com/twilio/twilio-node) from 3.63.0 to 3.63.1.
- [Release notes](https://github.com/twilio/twilio-node/releases)
- [Changelog](https://github.com/twilio/twilio-node/blob/main/CHANGES.md)
- [Commits](twilio/twilio-node@3.63.0...3.63.1)

---
updated-dependencies:
- dependency-name: twilio
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@sentry/integrations](https://github.com/getsentry/sentry-javascript) from 6.5.0 to 6.5.1.
- [Release notes](https://github.com/getsentry/sentry-javascript/releases)
- [Changelog](https://github.com/getsentry/sentry-javascript/blob/master/CHANGELOG.md)
- [Commits](getsentry/sentry-javascript@6.5.0...6.5.1)

---
updated-dependencies:
- dependency-name: "@sentry/integrations"
  dependency-type: direct:production
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [nocache](https://github.com/helmetjs/nocache) from 2.1.0 to 3.0.0.
- [Release notes](https://github.com/helmetjs/nocache/releases)
- [Changelog](https://github.com/helmetjs/nocache/blob/main/CHANGELOG.md)
- [Commits](helmetjs/nocache@v2.1.0...v3.0.0)

---
updated-dependencies:
- dependency-name: nocache
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [core-js](https://github.com/zloirock/core-js/tree/HEAD/packages/core-js) from 3.13.1 to 3.14.0.
- [Release notes](https://github.com/zloirock/core-js/releases)
- [Changelog](https://github.com/zloirock/core-js/blob/master/CHANGELOG.md)
- [Commits](https://github.com/zloirock/core-js/commits/v3.14.0/packages/core-js)

---
updated-dependencies:
- dependency-name: core-js
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [eslint](https://github.com/eslint/eslint) from 7.27.0 to 7.28.0.
- [Release notes](https://github.com/eslint/eslint/releases)
- [Changelog](https://github.com/eslint/eslint/blob/master/CHANGELOG.md)
- [Commits](eslint/eslint@v7.27.0...v7.28.0)

---
updated-dependencies:
- dependency-name: eslint
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [aws-sdk](https://github.com/aws/aws-sdk-js) from 2.920.0 to 2.922.0.
- [Release notes](https://github.com/aws/aws-sdk-js/releases)
- [Changelog](https://github.com/aws/aws-sdk-js/blob/master/CHANGELOG.md)
- [Commits](aws/aws-sdk-js@v2.920.0...v2.922.0)

---
updated-dependencies:
- dependency-name: aws-sdk
  dependency-type: direct:production
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
* chore(deps-dev): bump prettier from 2.2.1 to 2.3.1

Bumps [prettier](https://github.com/prettier/prettier) from 2.2.1 to 2.3.1.
- [Release notes](https://github.com/prettier/prettier/releases)
- [Changelog](https://github.com/prettier/prettier/blob/main/CHANGELOG.md)
- [Commits](prettier/prettier@2.2.1...2.3.1)

---
updated-dependencies:
- dependency-name: prettier
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

* ref: run npm lint

* ref: run npm run lint-html

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Kar Rui Lau <[email protected]>
)

* refactor(misc): removed unused parameter in submission factory; added typings to count submissions

* refactor(adminformservice): added method (and typing) for countFormSubmissions

* refactor(public): removed old Submission.count calls and replaced with countFormSubmissions

* test(adminformservice): adds test for countFormSubmissions

* feat(adminformservice): add new method (and typings) for retrieving form metadata

* test(adminformservice/test): adds tests for new method

* refactor(public): removed getMetadata and changed calls to getFormsMetadata

* feat(adminformservice): add new method (and type) for retrieval of single response

* test(adminformservice/test): adds test for getEncryptedResponse

* refactor(public): replaced Submission.getEncryptedResponse with new version

* chore(adminformservice): shifts submission related stuff to AdminSubmissionsService

* chore(public): updated prev callsites for submissions related methods

* fix(view-feedback.client.directive): removed duplicate services

* refactor(types/api/submission): changed typing of submission count query and associated methods

* refactor(getformsmetadata): split into 2 smaller methods and updated callsites

* test(adminsubmissionsservice): removed MockAxios import and uses jest.mock with mocked

using mockAxios impacted readability and understanding so it was changed to use jest.mock (which
mocks the whole module) together with mocked from ts-jest to mock return values

* refactor(adminsubmissionservice): renamed methods for greater clarity

* test(adminsubmissionsservice): changed test structure for increased clarity
…#2094)

Bumps [@typescript-eslint/parser](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/parser) from 4.26.0 to 4.26.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/parser/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v4.26.1/packages/parser)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/parser"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [typescript](https://github.com/Microsoft/TypeScript) from 4.2.4 to 4.3.2.
- [Release notes](https://github.com/Microsoft/TypeScript/releases)
- [Commits](microsoft/TypeScript@v4.2.4...v4.3.2)

---
updated-dependencies:
- dependency-name: typescript
  dependency-type: direct:development
  update-type: version-update:semver-minor
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Bumps [@typescript-eslint/eslint-plugin](https://github.com/typescript-eslint/typescript-eslint/tree/HEAD/packages/eslint-plugin) from 4.26.0 to 4.26.1.
- [Release notes](https://github.com/typescript-eslint/typescript-eslint/releases)
- [Changelog](https://github.com/typescript-eslint/typescript-eslint/blob/master/packages/eslint-plugin/CHANGELOG.md)
- [Commits](https://github.com/typescript-eslint/typescript-eslint/commits/v4.26.1/packages/eslint-plugin)

---
updated-dependencies:
- dependency-name: "@typescript-eslint/eslint-plugin"
  dependency-type: direct:development
  update-type: version-update:semver-patch
...

Signed-off-by: dependabot[bot] <[email protected]>

Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
mantariksh and others added 6 commits June 8, 2021 09:46
* chore: add script to delete endPage.buttons

* chore: remove endpage buttons from covid templates

* refactor: edit count statement and remove check for number of forms
* feat: log form updates

* chore: log formId, remove unknown typing for ControllerHandler

* chore: log req.query

* chore: remove end slash for route

Co-authored-by: Antariksh Mahajan <[email protected]>

Co-authored-by: Antariksh Mahajan <[email protected]>
* build(webhook-retries): create SQS queue in dev mode (1) (#1938)

* feat(webhook-retries): add isRetryEnabled to model (2) (#1939)

* feat(webhook-retries): produce and consume retries (3) (#1940)

* test(webhook-retries): add unit tests for retrieveWebhookInfoById (5) (#1942)

* test(webhook-retries): add unit tests for webhook service (6) (#1943)

* test(webhook-retries): add unit tests for message, producer, consumer (7) (#1988)

* test: add tests for WebhookQueueMessage

* test: add tests for webhook consumer

* test: add tests for WebhookProducer

* test: add tests for important utils

* feat: increase jitter with increasing base interval

* fix: calculate next attempt from time of initial attempt

* feat(webhook-retries): enable toggling retries on frontend (4) (#1941)

* test: make Date.now consistent in producer tests
@karrui
Copy link
Contributor Author

karrui commented Jun 8, 2021

Tests

@karrui
Don't do them yet, until staging VPC is up, @mantariksh will ping

Webhook retries

  • WEBHOOK_SQS_URL env var should be added in production.
  • Guide PR should be merged and go.gov.sg/form-webhook-retries should be updated to point to the correct section
  • Spin up a webhook server and set it up such that it returns a non-200 response for the first request, and subsequently returns 200. Enable retries on the form. Check in the logs that when the initial webhook fails, the webhook is added to the queue (message: "Pushed webhook to queue"). Check that the subsequent retry is received by your server, and the FormSG server logs "Webhook POST succeeded".
  • Without deactivating the form, disable retries. Ensure that your server is returning only non-200 responses, and submit the form. Check in the logs that the webhook fails but the webhook is not added to the queue (there is no "Pushed webhook to queue" log message).

@karrui
Copy link
Contributor Author

karrui commented Jun 8, 2021

Tests

@tshuli

Storage mode responses download autosort

  • Submit responses to storage mode form and download CSV. Ensure downloaded CSV is sorted (oldest to newest)

Email field model validator hook

  • Create email field directly in storage mode via API call by flipping includePdfSummary to true. Ensure that new email field is not created upon refresh.
  • Create email field in email mode. Ensure that including PDF summary is allowed.

@karrui
Copy link
Contributor Author

karrui commented Jun 8, 2021

@mantariksh

Tests

Submissions client partial refactor

Create a storage mode form with any field and activate it. Submit the form >= 11 times. Go to the data section and unlock responses.

  • There should be the correct number of responses displayed
  • Click on any page that is not 1. The relevant responses should be displayed
  • Select a time period without responses and attempt to download (both options). This should not work **
  • Select a time period with responses and attempt to download (both options). This should work
  • Search for a response using a valid referenceId. this should work and clicking into the response should display all related data.
  • Search for a response using an invalid/partial referenceId. There should be no responses displayed.
  • Clicking into a single response should work and display all data correctly.

@karrui
Copy link
Contributor Author

karrui commented Jun 8, 2021

@karrui

Tests

Prerequisites

  • Create email form with email field that has autoreply toggled
  • Create storage mode form with email field that has autoreply toggled

IE 11 public facing submission

  • Can access email form
  • Can submit email form
  • For email forms, should receive autoreply email (with pdf if toggled)
    • As admin, should receive submission email
  • For storage forms, should receive autoreply email
    • As admin, check responses tab and a new submission should have been made. (no need to be on IE11)
    • As admin, response should be decrypt-able (no need to be on IE11)
  • Should be able to submit feedback (check on admin side)

@tshuli
Copy link
Contributor

tshuli commented Jun 8, 2021

@mantariksh

Tests

Extra logging for form updates

  • Create a form with short text field. Edit the field. Check that the edit action shows up in the logs with details of the edit under body meta.

@karrui karrui force-pushed the release-v5.13.0 branch from 6fa6c88 to 4516bbc Compare June 8, 2021 07:01
@karrui karrui merged commit 11e004d into release Jun 8, 2021
@karrui karrui deleted the release-v5.13.0 branch August 17, 2021 02:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants