Emergency rollback of an app deployment
This document provides instructions on how to perform an emergency rollback. For non-severe issues, PRs should be submitted as normal.
These processes should primarily be done in production but can be practiced in other environments.
To rollback, you can pin the deployment-bag to a previous release. Simply select the release you want to pin to. In most cases, it will be the 2nd on the list. If you want to check different tags to see what images they are using, you can check the deployment-bag repo.
After pinning the release of the deployment-bag you want, simply start a new deploy.
The manual way to rollback an app without a full new app release is to log in to the AWS console as admin and roll back the Task Definition of the ECS Service corresponding to the app in question via clickops.
This may be needed in situations where a Concourse change has been made and Concourse has removed its deployment-bag history.
Ensure you pause the production pipeline before doing any manual interventions.
To do this, you can list Task Definitions from the top-level ECS page:
When choosing the Task Definition revision, you will probably need to change the listing's "Filter status" dropdown to show "Inactive" revisions, otherwise the most recent revisions won't be visible.
Once you've chosen a revision to roll back to, you'll need to "Create a new revision" based on it:
Once you've created the new revision the procedure to deploy it will be the same as in Deploying a demo app image to preview before merging
