From 23eb4771087dff56cf1ac237d822c461a41f668b Mon Sep 17 00:00:00 2001 From: Ludovic Delafontaine Date: Tue, 4 Apr 2023 09:00:40 +0200 Subject: [PATCH 1/3] Add support for Kubernetes arbitrary node selector documentation --- content/docs/ref/runner.md | 4 ++++ 1 file changed, 4 insertions(+) diff --git a/content/docs/ref/runner.md b/content/docs/ref/runner.md index 2307438a..b83a325a 100644 --- a/content/docs/ref/runner.md +++ b/content/docs/ref/runner.md @@ -60,6 +60,10 @@ Any [generic option](/doc/ref) in addition to: - `--cloud-aws-subnet=<...>`: [AWS subnet](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html#subnet-basics) identifier. +- `--cloud-kubernetes-node-selector=<...>`: `key=value` pair to specify the + [Kubernetes node selector](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/). + By default, the node selector is inferred from the GPU configuration. + [default: `accelerator=inferred`]. - `--docker-volumes=<...>`: Volume mount to pass to Docker, e.g. `/var/run/docker.sock:/var/run/docker.sock` for Docker-in-Docker support. May be specified multiple times. Only supported by GitLab. From 4c127f510c9433f507653d0acff0954e757fccb2 Mon Sep 17 00:00:00 2001 From: Ludovic Delafontaine Date: Tue, 11 Apr 2023 11:31:53 +0200 Subject: [PATCH 2/3] Kubernetes node selector is now an array in documentation and add examples --- content/docs/ref/runner.md | 57 ++++++++++++++++++++++++++++++++++++-- 1 file changed, 54 insertions(+), 3 deletions(-) diff --git a/content/docs/ref/runner.md b/content/docs/ref/runner.md index b83a325a..28ce5407 100644 --- a/content/docs/ref/runner.md +++ b/content/docs/ref/runner.md @@ -61,9 +61,10 @@ Any [generic option](/doc/ref) in addition to: [AWS subnet](https://docs.aws.amazon.com/vpc/latest/userguide/VPC_Subnets.html#subnet-basics) identifier. - `--cloud-kubernetes-node-selector=<...>`: `key=value` pair to specify the - [Kubernetes node selector](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/). - By default, the node selector is inferred from the GPU configuration. - [default: `accelerator=inferred`]. + Kubernetes node selector. May be + [specified multiple times](http://yargs.js.org/docs/#array). More + [details below](#using---cloud-kubernetes-node-selector). [default: + `accelerator=infer`] - `--docker-volumes=<...>`: Volume mount to pass to Docker, e.g. `/var/run/docker.sock:/var/run/docker.sock` for Docker-in-Docker support. May be specified multiple times. Only supported by GitLab. @@ -376,6 +377,56 @@ Azure. replacing the `IP_ADDRESS` placeholder with the instance address returned by `cml runner` (search the output logs for `instanceIp`). +### Using `--cloud-kubernetes-node-selector` + +Set the +[Kubernetes node selector](https://kubernetes.io/docs/tasks/configure-pod-container/assign-pods-nodes/). + +For example: + +```cli +$ cml runner launch \ + --cloud-kubernetes-node-selector="disktype=ssd" \ + ... +``` + +will select the node labeled with `disktype=ssd`. + +#### Node selector on multiple labels + +You can set multiple labels for a node selector. + +For example: + +```cli +$ cml runner launch \ + --cloud-kubernetes-node-selector="disktype=ssd" \ + --cloud-kubernetes-node-selector="ram=huge" \ + ... +``` + +will select the node labeled with `disktype=ssd` and `ram=huge`. + +If you specify the same key multiple times, the last one will be used. + +#### Infer the value from the GPU configuration + +If you set the key value to `infer`, it will infer the GPU type from the GPU +configuration on the key you have set. + +For example: + +```cli +$ cml runner launch \ + --cloud-kubernetes-node-selector="accelerator=infer" \ + ... +``` + +will select the node labeled `accelerator` with the value inferred from the GPU +configuration if available, e.g. `k80`. + +By default, the key to infer on is `accelerator`. + ## Debugging [See the section self-hosted runners](/doc/self-hosted-runners#debugging) From 14dfd32fe195fbbf3b6105292e0c860b3d01b705 Mon Sep 17 00:00:00 2001 From: Ludovic Delafontaine Date: Mon, 17 Apr 2023 11:40:02 +0200 Subject: [PATCH 3/3] Improve information regarding the default value --- content/docs/ref/runner.md | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/content/docs/ref/runner.md b/content/docs/ref/runner.md index 28ce5407..55130e17 100644 --- a/content/docs/ref/runner.md +++ b/content/docs/ref/runner.md @@ -392,6 +392,8 @@ $ cml runner launch \ will select the node labeled with `disktype=ssd`. +If not provided, a default `accelerator=infer` key pair will be used. + #### Node selector on multiple labels You can set multiple labels for a node selector. @@ -418,15 +420,13 @@ For example: ```cli $ cml runner launch \ - --cloud-kubernetes-node-selector="accelerator=infer" \ + --cloud-kubernetes-node-selector="gpu=infer" \ ... ``` -will select the node labeled `accelerator` with the value inferred from the GPU +will select the node labeled `gpu` with the value inferred from the GPU configuration if available, e.g. `k80`. -By default, the key to infer on is `accelerator`. - ## Debugging [See the section self-hosted runners](/doc/self-hosted-runners#debugging)