add required permissions for uploading image to a given bucket

README.md

@@ -14,17 +14,78 @@ The repositories listed below contain the definitions of the required components
 
 For convenience, an [installation script](https://raw.githubusercontent.com/antmicro/runner/vm-runners/scripts/install.sh) is available that installs dependencies, configures the system, clones the repository and builds the runner.
 
+
 ## Installation and configuration
 
-The manual below assumes that Debian Buster is used to deploy the runner.
+### catx-ext-umich specific procedures
+
+Start from scratch by following the procedure below:
+
+``` bash
+gcloud compute instances stop instance-2
+
+gcloud compute instances delete instance-2
+
+gcloud compute instances create instance-2 \
+--project=catx-ext-umich \
+--zone=us-central1-a \
+--machine-type=e2-standard-32 \
+--network-interface=network-tier=PREMIUM,subnet=default \
+--maintenance-policy=MIGRATE \
+--service-account=terraform-runner@catx-ext-umich.iam.gserviceaccount.com \
+--scopes=https://www.googleapis.com/auth/cloud-platform \
+--create-disk=auto-delete=yes,boot=yes,device-name=instance-2,\
+image=projects/debian-cloud/global/images/debian-10-buster-v20210916,\
+mode=rw,size=256,type=projects/catx-ext-umich/zones/us-central1-a/diskTypes/pd-balanced \
+--no-shielded-secure-boot \
+--shielded-vtpm \
+--shielded-integrity-monitoring \
+--reservation-affinity=any
+
+# Delete previous image archive in bucket
+# TODO: How to delete uploaded image (Not just in bucket)
+gsutil rm gs://catx-ext-umich-worker-bucket/scalenode-9e1d63d.tar.gz
+
+# Connect to refreshed instance
+gcloud compute ssh instance-2 --ssh-flag="-ServerAliveInterval=30"
+```
+
+Before running `setup.sh` make sure that the following is done:
+- Install `git` upon ssh to the created instance
+- The archive in the bucket and corresponding image is deleted
+    - Screenshot to-be included
+- Fill in the `TOKEN` in `coor.sh`
+    - The `TOKEN` can be found here: https://github.com/idea-fasoc/OpenFASOC/settings/actions/runners/new?arch=x64&os=linux under `configure`. The `TOKEN` expires in a hour.
+
+After everything is setup, run `setup.sh` to build everything from scratch. (This script assumes that all virtual infrastructure created by Terraform does not need to be re-configured.)
 
 ### Host prerequisites
 
+The manual below assumes that Debian Buster is used to deploy the runner.
+
 The following packages must be installed:
 
 * `build-essential`
 * [Terraform](https://www.terraform.io/docs/cli/install/apt.html)
+```bash
+sudo apt-get install software-properties-common
+
+curl -fsSL https://apt.releases.hashicorp.com/gpg | 
+    sudo apt-key add -
+
+sudo apt-add-repository "deb [arch=$(dpkg --print-architecture)] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
+```
 * [Google Cloud SDK](https://cloud.google.com/sdk/docs/install#deb)
+```bash
+echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | 
+    sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
+
+sudo apt-get install apt-transport-https ca-certificates gnupg
+curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | 
+    sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
+
+sudo apt-get update && sudo apt-get install google-cloud-sdk
+```
 
 ### Installation steps
 
@@ -46,22 +107,24 @@ export SERVICE_ACCOUNT_ID=runner-manager
 gcloud iam service-accounts create $SERVICE_ACCOUNT_ID
 
 gcloud projects add-iam-policy-binding $PROJECT \
-    --member="serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT \
+    --member="serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT.iam.gserviceaccount.com" \
     --role="roles/compute.admin"
 
 gcloud projects add-iam-policy-binding $PROJECT \
-    --member="serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT \
+    --member="serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT.iam.gserviceaccount.com" \
     --role="roles/iam.serviceAccountCreator"
 
 gcloud projects add-iam-policy-binding $PROJECT \
-    --member="serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT \
+    --member="serviceAccount:$SERVICE_ACCOUNT_ID@$PROJECT.iam.gserviceaccount.com" \
     --role="roles/iam.serviceAccountUser"
 
 # Create and download SA key.
 # WARNING: the export below will be used by Terraform later.
-export GOOGLE_APPLICATION_CREDENTIALS
+# This command is for the OUTPUT_FILE option for "keys create"
+export GOOGLE_APPLICATION_CREDENTIALS=~/key.out
+
 gcloud iam service-accounts keys create $GOOGLE_APPLICATION_CREDENTIALS \
-    --iam-account=$SERVICE_ACCOUNT_ID@$PROJECT
+    --iam-account=$SERVICE_ACCOUNT_ID@$PROJECT.iam.gserviceaccount.com
 
 # Create a GCP bucket for worker image.
 export BUCKET=$PROJECT-worker-bucket
@@ -72,21 +135,55 @@ Build and upload the worker image:
 
 ```bash
 # Clone the repository
-git clone https://github.com/antmicro/github-actions-runner-scalerunner.git
+git clone --recursive https://github.com/antmicro/github-actions-runner-scalerunner.git
 cd github-actions-runner-scalerunner
 
 # Compile bzImage
 cd buildroot && make BR2_EXTERNAL=../overlay/ scalenode_gcp_defconfig && make
 
 # Prepare a disk for GCP
 ./make_gcp_image.sh
+```
 
-# Upload the resulting tar archive
-./upload_gcp_image.sh $PROJECT $BUCKET
+### Adjust Service Account priviliges before uploading built disk to GCP
+
+Save the bucket's IAM policy to a temporary (arbitrary) JSON file
+```bash
+gsutil iam get gs://$BUCKET > /arbitrary/path/file.json
+```
+Get the project name and default service account email address. Adjust filter accordingly if a different service account is used
+```bash
+export PROJECT=$(gcloud config get-value project)
+export SA=$(gcloud iam service-accounts list --filter=default | 
+    grep -E -o '[a-z0-9._%+-]+@[a-z0-9.-]+(\.[a-z0-9._%+-]+)?[a-z]{2,4}')
+```
+Get the absolute path of the Bucket config file
+```bash
+export BUCKET_FILE=/arbitrary/path/file.json
 ```
+Using the `sed` utility to insert required permissions associated with the bucket
+```bash
+sed -i 's/"bindings": \[/"bindings": \[\
+    {\
+      "members": \[\
+        "projectEditor:'"$PROJECT"'",\
+        "projectOwner:'"$PROJECT"'",\
+        "serviceAccount:'"$SA"'"\
+      \],\
+      "role": "roles\/storage.legacyBucketOwner"\
+    \},/' $BUCKET_FILE
+```
+Upload the modified bucket file back to GCloud
+```bash
+gsutil iam set $BUCKET_FILE gs://$BUCKET
+```
+Upload the resulting tar archive
 
-Setup virtual infrastructure using Terraform:
+```bash
+./upload_gcp_image.sh $PROJECT $BUCKET
+```
 
+### Setup virtual infrastructure using Terraform:
 ```bash
 git clone https://github.com/antmicro/github-actions-runner-terraform.git
 terraform init && terraform apply
@@ -95,7 +192,7 @@ terraform init && terraform apply
 Connect to the coordinator instance created in the previous step:
 
 ```bash
-gcloud compute --zone <COORDINATOR_ZONE> ssh <COORDINATOR_INSTANCE>
+gcloud compute ssh gha-runner-coordinator --zone=us-west1-a
 ```
 
 Install and configure the runner on the coordinator instance:

coor.sh

@@ -0,0 +1,12 @@
+sudo -i -u runner bash
+cd ~/github-actions-runner
+
+export REPOSITORY_ORG=idea-fasoc \
+    export REPOSITORY_NAME=OpenFASOC \
+    export TOKEN= \
+    export SLOTS=1 \
+    export SCALE=1
+
+./config.sh --url https://github.com/$REPOSITORY_ORG/$REPOSITORY_NAME \
+            --token $TOKEN \
+            --num $SLOTS

setup.sh

@@ -0,0 +1,50 @@
+# Setup prerequisites
+sudo apt install build-essential -y
+sudo apt install git wget unzip rsync bc \
+    libelf-dev autotools-dev automake \
+    gcc-multilib texinfo dosfstools mtools -y
+
+# Install Go for compiling BzImage
+cd ~
+wget https://golang.org/dl/go1.16.5.linux-amd64.tar.gz && \
+    sudo tar -xzf go1.16.5.linux-amd64.tar.gz -C /usr/local/ && \
+    export PATH=$PATH:/usr/local/go/bin
+
+# Install Terraform
+sudo apt-get install software-properties-common -y
+curl -fsSL https://apt.releases.hashicorp.com/gpg | 
+    sudo apt-key add -
+sudo apt-add-repository "deb [arch=$(dpkg --print-architecture)] https://apt.releases.hashicorp.com $(lsb_release -cs) main"
+sudo apt update && sudo apt install terraform -y
+
+# Install Google Cloud SDK
+echo "deb [signed-by=/usr/share/keyrings/cloud.google.gpg] https://packages.cloud.google.com/apt cloud-sdk main" | 
+    sudo tee -a /etc/apt/sources.list.d/google-cloud-sdk.list
+
+sudo apt-get install apt-transport-https ca-certificates gnupg -y
+curl https://packages.cloud.google.com/apt/doc/apt-key.gpg | 
+    sudo apt-key --keyring /usr/share/keyrings/cloud.google.gpg add -
+
+sudo apt-get update && sudo apt-get install google-cloud-sdk -y
+
+# Rebuild image
+git clone --recursive \
+    https://github.com/antmicro/github-actions-runner-scalerunner.git && \
+        cd github-actions-runner-scalerunner/buildroot && \
+        make BR2_EXTERNAL=../overlay/ scalenode_gcp_defconfig && \
+        make
+
+export PROJECT=catx-ext-umich && \
+    export BUCKET=$PROJECT-worker-bucket
+
+# Make and upload image
+cd ../ && \
+    ./make_gcp_image.sh && \
+    ./upload_gcp_image.sh $PROJECT $BUCKET
+
+# ssh into coordinator instance to setup runner and run conifg
+cd ~/runner
+export name=$(gcloud compute instances list | grep gha | awk '{print $1}') && \
+    export zone=$(gcloud compute instances list | grep gha | awk '{print $2}') && \
+    cat coor.sh | gcloud compute ssh $name --zone=$zone
+