Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: automate critical bounce handling #318

Merged
merged 68 commits into from
Sep 22, 2020
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
68 commits
Select commit Hold shift + click to select a range
83c2019
feat: change hasAlarmed to hasEmailed
mantariksh Sep 14, 2020
119fbb5
refactor: rename logCriticalBounce
mantariksh Sep 14, 2020
60a5995
refactor: change bounceType to enum
mantariksh Sep 14, 2020
cf3abad
refactor: convert handleCriticalBounce to async
mantariksh Sep 14, 2020
849841f
feat: implement getEmails
mantariksh Sep 14, 2020
08ad268
feat: implement skeleton for autoemails
mantariksh Sep 14, 2020
8d90367
feat: add email templates
mantariksh Sep 14, 2020
6817491
refactor: move mail types into src/types
mantariksh Sep 14, 2020
7e200dd
feat: implement sendBounceNotification
mantariksh Sep 14, 2020
8f903cb
feat: send email for critical bounces
mantariksh Sep 14, 2020
f45c9f2
refactor: use _.difference
mantariksh Sep 14, 2020
74ca1f6
feat: log auto-email recipients
mantariksh Sep 14, 2020
b4b28c9
docs: edit jsdocs for sendBounceNotification
mantariksh Sep 14, 2020
954a79d
feat: include submissionId as email header
mantariksh Sep 14, 2020
08f92a7
feat: implement deactivate method
mantariksh Sep 14, 2020
5c4ab84
feat: deactivate form for permanent bounce
mantariksh Sep 14, 2020
280cb7f
test: replace hasAlarmed with hasEmailed
mantariksh Sep 14, 2020
4c6c3bf
fix: allow for undefined submissionId
mantariksh Sep 14, 2020
3b34b7c
fix: fix types import
mantariksh Sep 14, 2020
c3b8971
test: fix test imports
mantariksh Sep 14, 2020
87c6e02
test: add tests for mail service
mantariksh Sep 15, 2020
f23bf72
refactor: remove unused import for TODO
mantariksh Sep 15, 2020
62a802e
test: fix subject in mail test
mantariksh Sep 15, 2020
cac9204
feat: return regular Array from getEmails
mantariksh Sep 15, 2020
18a9afd
test: add tests for auto emails
mantariksh Sep 15, 2020
c63face
refactor: rename snsService to BounceService
mantariksh Sep 16, 2020
4d69e9f
feat: store bounceType in database
mantariksh Sep 16, 2020
01ebb29
feat: store bounceType from notification
mantariksh Sep 16, 2020
1dfa65d
refactor: implement util functions for email status
mantariksh Sep 16, 2020
b26c421
feat: update bounceType when merging bounce docs
mantariksh Sep 16, 2020
54284b8
feat: implement areAllPermanentBounces
mantariksh Sep 16, 2020
19719fb
feat: perform perm actions only if all are perm
mantariksh Sep 16, 2020
9e5fc2f
refactor: return early if no action required
mantariksh Sep 16, 2020
96352be
refactor: convert deactivate to static method
mantariksh Sep 16, 2020
9aca961
feat: remove submissionId from notif email
mantariksh Sep 16, 2020
4fdab9a
feat: implement updateHasEmailed
mantariksh Sep 16, 2020
b518d43
refactor: transfer more control to controller
mantariksh Sep 16, 2020
8c7c9ba
fix: log with latest hasEmailed
mantariksh Sep 17, 2020
88ac039
refactor: remove unused import
mantariksh Sep 17, 2020
3155ab6
test: add some tests for bounce controller
mantariksh Sep 17, 2020
0cde83b
refactor: simplify logic for domain methods
mantariksh Sep 17, 2020
cceb364
refactor: transfer more control to controller
mantariksh Sep 17, 2020
7f4ede4
refactor: rename updateHasEmailed to setHasEmailed
mantariksh Sep 17, 2020
85057a3
test: add and update bounce model tests
mantariksh Sep 18, 2020
686e114
feat: log number of permanent and transient for critical bounce
mantariksh Sep 18, 2020
0f4d313
feat: add safety check in deactivateById
mantariksh Sep 18, 2020
d0534fd
test: add tests for deactivateById
mantariksh Sep 18, 2020
7258bc0
refactor: rename hasEmailed to hasAutoEmailed
mantariksh Sep 18, 2020
ebfd562
feat: update meta.action
mantariksh Sep 18, 2020
8478296
test: update mail service tests
mantariksh Sep 18, 2020
4392f52
feat: log verification OTP to short-term logs
mantariksh Sep 18, 2020
62ffb4b
test: update tests for Bounce service
mantariksh Sep 18, 2020
82194ec
feat: enable form deactivation
mantariksh Sep 18, 2020
0848690
test: update tests for bounce controller
mantariksh Sep 18, 2020
af94e45
test: update tests for bounce controller
mantariksh Sep 18, 2020
054dba2
test: fix tests for bounce service
mantariksh Sep 18, 2020
a5a1516
test: fix tests for bounce model
mantariksh Sep 18, 2020
b12c7b8
test: fix form model tests
mantariksh Sep 18, 2020
2a5dad2
test: fix tests for mail service
mantariksh Sep 18, 2020
efdebf5
test: add test for nth call in mail service
mantariksh Sep 18, 2020
222143f
fix: use relative imports
mantariksh Sep 18, 2020
dbdf595
refactor: move deactivateForm into form service
mantariksh Sep 21, 2020
5b443fd
feat: implement and test hasNotified
mantariksh Sep 21, 2020
1422784
refactor: rename setHasAutoEmailed
mantariksh Sep 21, 2020
67920e8
refactor: move hasNotified check into controller
mantariksh Sep 21, 2020
80d4714
refactor: change log message as requested
mantariksh Sep 21, 2020
7b5825c
feat: change BOUNCE_LIFE_SPAN to 24h
mantariksh Sep 21, 2020
6bd08c8
feat: log deactivation status
mantariksh Sep 22, 2020
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .template-env
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ FORMSG_SDK_MODE=
## App Config
# APP_NAME=FormSG
# OTP_LIFE_SPAN=900000
# BOUNCE_LIFE_SPAN=1800000
# BOUNCE_LIFE_SPAN=86400000
# AGGREGATE_COLLECTION=

# If provided, a banner with the provided message will show up in every form.
Expand Down
2 changes: 1 addition & 1 deletion docker-compose.yml
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ services:
- IMAGE_S3_BUCKET=local-image-bucket
- LOGO_S3_BUCKET=local-logo-bucket
- FORMSG_SDK_MODE=development
- BOUNCE_LIFE_SPAN=1800000
- BOUNCE_LIFE_SPAN=86400000
- AWS_ACCESS_KEY_ID=fakeKey
- AWS_SECRET_ACCESS_KEY=fakeSecret
- SESSION_SECRET=thisisasecret
Expand Down
80 changes: 40 additions & 40 deletions docs/DEPLOYMENT_SETUP.md

Large diffs are not rendered by default.

14 changes: 14 additions & 0 deletions src/app/models/form.server.model.ts
Original file line number Diff line number Diff line change
Expand Up @@ -89,6 +89,7 @@ const formSchemaOptions: SchemaOptions = {
export interface IFormModel extends Model<IFormSchema> {
getOtpData(formId: string): Promise<FormOtpData | null>
getFullFormById(formId: string): Promise<IPopulatedForm | null>
deactivateById(formId: string): Promise<IFormSchema | null>
}

type IEncryptedFormModel = Model<IEncryptedFormSchema>
Expand Down Expand Up @@ -469,6 +470,19 @@ const compileFormModel = (db: Mongoose): IFormModel => {
return data
}

// Deactivate form by ID
FormSchema.statics.deactivateById = async function (
this: IFormModel,
formId: string,
): Promise<IFormSchema | null> {
const form = await this.findById(formId)
if (!form) return null
if (form.status === Status.Public) {
form.status = Status.Private
}
return form.save()
}

// Hooks
FormSchema.pre<IFormSchema>('validate', function (next) {
// Reject save if form document is too large
Expand Down
69 changes: 44 additions & 25 deletions src/app/modules/bounce/__tests__/bounce-test-helpers.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,9 @@
import { ObjectId } from 'bson'
import { cloneDeep, merge, pick } from 'lodash'

import { EmailType } from 'src/app/constants/mail'
import {
BounceType,
IBounce,
IBounceNotification,
IBounceSchema,
Expand All @@ -26,7 +28,7 @@ const makeEmailNotification = (
formId: ObjectId,
submissionId: ObjectId,
recipientList: string[],
emailType: 'Admin (response)' | 'Email confirmation',
emailType: EmailType,
): IEmailNotification => {
return {
notificationType,
Expand Down Expand Up @@ -56,15 +58,27 @@ const makeEmailNotification = (
}
}

export const makeBounceNotification = (
formId: ObjectId = new ObjectId(),
submissionId: ObjectId = new ObjectId(),
recipientList: string[] = [],
bouncedList: string[] = [],
bounceType: 'Transient' | 'Permanent' = 'Permanent',
emailType: 'Admin (response)' | 'Email confirmation' = 'Admin (response)',
): ISnsNotification => {
const Message = merge(
export const makeBounceNotification = ({
formId,
submissionId,
recipientList,
bouncedList,
bounceType,
emailType,
}: {
formId?: ObjectId
submissionId?: ObjectId
recipientList?: string[]
bouncedList?: string[]
bounceType?: BounceType
emailType?: EmailType
} = {}): IBounceNotification => {
formId ??= new ObjectId()
submissionId ??= new ObjectId()
recipientList ??= []
bouncedList ??= []
emailType ??= EmailType.AdminResponse
return merge(
makeEmailNotification(
'Bounce',
formId,
Expand All @@ -81,19 +95,27 @@ export const makeBounceNotification = (
},
},
) as IBounceNotification
const body = cloneDeep(MOCK_SNS_BODY)
body.Message = JSON.stringify(Message)
return body
}

export const makeDeliveryNotification = (
formId: ObjectId = new ObjectId(),
submissionId: ObjectId = new ObjectId(),
recipientList: string[] = [],
deliveredList: string[] = [],
emailType: 'Admin (response)' | 'Email confirmation' = 'Admin (response)',
): ISnsNotification => {
const Message = merge(
export const makeDeliveryNotification = ({
formId,
submissionId,
recipientList,
deliveredList,
emailType,
}: {
formId?: ObjectId
submissionId?: ObjectId
recipientList?: string[]
deliveredList?: string[]
emailType?: EmailType
} = {}): IDeliveryNotification => {
formId ??= new ObjectId()
submissionId ??= new ObjectId()
recipientList ??= []
deliveredList ??= []
emailType ??= EmailType.AdminResponse
return merge(
makeEmailNotification(
'Delivery',
formId,
Expand All @@ -107,9 +129,6 @@ export const makeDeliveryNotification = (
},
},
) as IDeliveryNotification
const body = cloneDeep(MOCK_SNS_BODY)
body.Message = JSON.stringify(Message)
return body
}

// Omit mongoose values from Bounce document
Expand All @@ -118,7 +137,7 @@ export const extractBounceObject = (
): Omit<IBounce, '_id'> => {
const extracted = pick(bounce.toObject(), [
'formId',
'hasAlarmed',
'hasAutoEmailed',
'expireAt',
'bounces',
])
Expand Down
Loading