CI & staging

How the pipeline works

1. Developers make changes to the main branch on Github.

2. The change in the main-branch triggers a GitHub Action, which results in a Docker image being built and getting pushed to a DockerHub repository.

   (On the current staging server:)

3. A Watchtower container running on the staging server keeps track of new Docker images on DockerHub. It will pull the uploaded images when it detects changes and restarts containers, resulting in the updated images being run with minimal delay. The updating should thus work automagically.

4. The changes are now visible on the staging server (see that some changes like changes to the database might need more than just restarting. In other words, manual actions on the server. See Useful and needed docker commands.)

Setting up the pipeline

1. Create a Docker Hub account and create an access token. An access token can be created by logging in to Docker Hub and clicking on your username in the top right corner and selecting Account Settings > Security > New Access Token.
2. Provide DOCKERHUB_USERNAME and DOCKERHUB_TOKEN as Github Secrets. The secrets cannot be read after they have been set so they are safe from your fellow developers!
   • DOCKERHUB_USERNAME is your Docker Hub username
   • DOCKERHUB_TOKEN is the access token you created

Setting up the staging server

1. Ask instructors for permissions to access the staging server.
2. Login to the staging server according to the instructions you should have been provided with by the instructors. At the moment of writing, this includes running a login script in the root folder.
3. Create a file docker-compose.yml in /home/ohtup_user/qleader/. Here is an example docker-compose.yml based on the current file in the staging.
4. Create a file qleader.subfolder.conf in /home/ohtup_user/nginx/config/nginx/proxy-confs/ like this configuration
5. Create a .env file with all the OAuth2 secrets and other key information (example .env)
   • For how to get the keys and secrets see our Python social auth page
6. Create a folder /home/ohtup_user/qleader/static which holds the Django and REST Framework static files. Lighttpd in container qleader-static will serve these.

If the models change and the staging server returns an error message

If the data structure changes in the program, but the database itself is left unchanged, the mismatch will result in an error, and the service will not be able to work. At the moment of writing this, our solution for this has been to delete the old database and build a new one (if the program is in production, there would need to be a way to migrate the data from the old db to the new one -- we don't know how to do this yet).

The simplest way to delete the database on the staging server ("ohtup-staging") is to stop and remove the docker container. See the Docker cheat sheet

Known issues

The staging server runs in a path /qleader so hard-coded links between pages will not work. For example, this is NOT going to work:

return redirect('/newMolecule/')  # WILL NOT WORK ON STAGING!

You should use a view name instead (defined in urls.py) so this is correct:

return redirect('newMolecule')  # using view name defined in urls.py

In HTML templates you can use the variable path_prefix:

<a href="{{ path_prefix }}/newMolecule/">New molecule</a>

---