Deployment on Openshift

Deploying on OpenShift allows you to manage the deployment from your own computer

Note

This procedure will require you to be in CERN's network, or to have tunneled your CERN traffic through lxtunnel. To do so, you can use sshuttle. Instructions here.

Requirements

• docker installed on your computer or on LXPLUS, in order to build the docker images:
  • The frontend is pushed to cms-dqmdc/runregistry-frontend
  • The backend is pushed to cms-dqmdc/runregistry-backend
• The oc (Openshift commandline) tool. Instructions on how to download it on your computer can be found here.
• A userkey.pem and a usercert.pem file, which you can get by following the procedure here.

Overview

The main functionality of Run Registry runs in two containers: runregistry-backend and runregistry-frontend. Those containers are built locally and pushed to CERN's image registry (registry.cern.ch).

The main route to the application is managed by an nginx server, which is accompanied by an oauth2-proxy, for authenticating.

The backend requires a redis instance, for managing the queues for the JSON creation.

An rough overview of the services and their interactions can be seen below:

graph TD;

subgraph Openshift
    nginx(nginx)
    sso(CERN SSO
    Proxy)
    subgraph RunRegistry
    be(Backend@9500)
    fe(Frontend@7001)
    end
    redis(Redis)
end

nginx <-->|/auth| sso
nginx <-->|/api| be 
nginx <-->|/*| fe
be <--> redis

Loading

Procedure for the first deployment

Creating a new Web Service

1. Head over to CERN's Web Services Portal and create a new PaaS Project:

The project should now have been created, and you can manage it using the oc utility, and the PaaS website.

Add Redis

1. Open and select your project on PaaS.

2. Navigate to Developer > Topology
3. Right click in the empty space and Add to Project > From Catalog.

4. Select Redis, and click Instantiate Template.

5. Leave all fields to the default values, but select version 5-el8 (the newer version do not seem to work properly):

Note

This procedure also creates a secret to connect to the Redis instance. You can find it under Administrator > Workloads > Secrets > redis

Add the SSO Proxy

1. Open and select your project on PaaS.
2. Navigate to Developer > Topology
3. Right click in the empty space and Add to Project > Helm Charts.
4. Select Cern Auth Proxy and Create.

5. In the Create Helm Release form, select the latest Chart version.
6. Then select YAML view and paste the contents of the deployment/prod/sso_proxy_helmchart.yaml
7. Click on Create.

Note

By default, the SSO proxy will use the oidc-client-secret which is automatically created upon PaaS project creation. If you need to use a manually created SSO Application Registration, you will need to modify this secret to use the Client ID and Secrets of the custom SSO Application Registration.

Build and push the docker images

1. Clone this repository to your computer.
2. Go to CERN image registry and click on your profile > User profile:

3. Copy the CLI secret:

4. On your computer, run:

docker login registry.cern.ch

Use your CERN account name and the CLI secret you copied.

5. Build and push the backend image:

cd runregistry_backend
sudo docker build -t registry.cern.ch/cms-dqmdc/runregistry-backend . --network=host
sudo docker push registry.cern.ch/cms-dqmdc/runregistry-backend

6. Build and push the frontend image:

cd runregistry_frontend
sudo docker build -t registry.cern.ch/cms-dqmdc/runregistry-frontend . --network=host
sudo docker push registry.cern.ch/cms-dqmdc/runregistry-frontend

Import the images to your project

1. Go to PaaS and copy the login command:

2. Run the command you copied in a terminal on your computer (requires oc).
3. Select your PaaS project by name

oc project <your project name>

4. Import the two docker images you uploaded:

oc import-image runregistry-backend --from=registry.cern.ch/cms-dqmdc/runregistry-backend --confirm
oc import-image runregistry-frontend --from=registry.cern.ch/cms-dqmdc/runregistry-frontend --confirm

Deploy the secrets

Three secrets are required for the deployment:

• The runregistry-backend-secret, for configuring the backend,
• The runregistry-frontend-secret, for configuring the frontend and
• The grid-cert-secret, for storing a grid certificate, used for accessing DQMGUI.

The secret files supplied in this repository (deployment/prod/secrets) are empty by default, and you will have to fill them by hand (or script) during this step. If you're going to change the files directly, you will have to encode the values with base64 (e.g. echo <my secret value> -n | base64). Else, you can leave them empty, and change the values from the PaaS website.

1. Apply the secrets:

oc apply -f deployment/prod/secrets

2. Verify that the secrets were applied by going to the PaaS website and navigating to Administrator > Workloads > Secrets. They should all have keys but no values listed, when selecting a secret and clicking Actions > Edit Secret.

2. Edit the secrets. Modify the values as follows, and then click Save:

• Backend

Key	Description
DB_HOSTNAME	Database hostname
DB_USERNAME	Database username
DB_PASSWORD	Database password
DB_NAME	The name of the database to use in DB_HOSTNAME
DB_PORT	Database port, namely the one for the DBoD instance used for Run Registry
REDIS_HOST	The name of the Redis you deployed earlier. It should be redis.
REDIS_PORT	The port of the Redis you deployed earlier. It should be 6379 by default.
ENV	Selects a preset of configuration values found in config.js. Put prod_kubernetes for production.
NODE_ENV	Put production.

• Frontend

Key	Description
ENV	Selects a preset of configuration values found in config.js. Put kubernetes for production.
NODE_ENV	Put production.

• Grid certificate

Key	Description
usercert.pem	Paste the contents of a valid usercert.pem file.
userkey.pem	Paste the contents of a valid userkey.pem file.

Deploy the ConfigMaps

oc apply -f deployment/prod/configmaps

Deploy the actual Deployments

oc apply -f deployment/prod/deployments

Deploy the Services

oc apply -f deployment/prod/services

Deploy the Routes

oc apply -f deployment/prod/routes.yaml

Restart if needed

oc rollout restart deployment/nginx
oc rollout restart deployment/runregistry-backend
oc rollout restart deployment/runregistry-frontend

Update the application permissions

1. Navigate to CERN's Application Portal
2. Find the registration for your application. Its name should be something like webframeworks-paas-<your application name>.
3. Run the scripts/runregistry_sso_manager.py script to update the SSO registration and add all the required roles, mapped to e-groups. You will need either a bearer token (get one by authenticating here) and using it with --token, or by using the --client-id and --client-secret of an SSO application authorized to manage the Run Registry's SSO registration.

Warning

The client-id and client-secret method does not seem to work properly for some reason, and it can only read, not write through the authentication API.

Redeploying

Manually

Once the first deployment is complete, and assuming you only want to modify the docker images, you will only need to re-run the following steps:

1. Rebuild and push the docker images to registry.cern.ch (here).
2. Import the images to PaaS again (here)
3. Restart the services (here)

Using npm commands

For convenience, npm commands have been added to the backend's and frontend's package.json.

Backend:

cd runregistry_backend
oc login  # Replace with the command with paas.cern.ch
oc project dev-cmsrunregistry  # Remove "dev-" to redeploy the production instance
docker login registry.cern.ch  # You will be asked for the registry.cern.ch username and password
npm run docker_build_and_push --mytag <new tag>  # Replace <new tag> with the new version you are uploading, keep the vX.Y.Z convention
npm run openshift_import_and_restart  # The deployment will automatically get the latest image you just uploaded

Frontend:

cd runregistry_frontend
# The exact same commands.