Update deploy task to warn devs of migration status after a failed deploy

[exalate-issue-sync]

During the API deploy process, it is possible for migrations to be all or partially applied even when the deploy fails. While this doesn’t need to cause downtime (so long as the migrations were backwards compatible), it still leaves the API in a somewhat delicate state. (See Intra-Deploy Scenario #5 in our docs here)

At the end of the deploy process, devs should be notified in the logs as to which migrations were applied. In the event that the deploy process failed, this should be part of a clear, highly visible postmortem log entry.

QA Notes

No external changes will be visible. Devs should provide screenshots showing examples of the new logs.

DEV Notes

The deploy process should not delete the migrator app until after the deploy process has finished.

After the deploy process has finished, use {{showmigrations --list}} (run against the migrator and not fecfile-web-api) to determine which migrations were successfully applied. In the deploy process, you can use {{ctx.run().stdout}} to retrieve the stdout stream from the running task, which might be helpful.

Write an end of deploy summary for the logs, ideally in a separate CircleCI step.

this should include any failures and information about what failed, specifically

it should include a list of migrations that were successfully applied and a list of any that failed

Determine the ease of accessing fecfile-api-migrator logs through Kibana. If it’s difficult, then we should stop deleting the migrator app. Investigate whether using the {{cf stop [APP]}} command saves memory and/or CPU resources.

Design

null

See full ticket and images here: FECFILE-1957