Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add simple queue control example to python docs #967

Merged
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 62 additions & 1 deletion site/content/en/docs/tasks/run_python_jobs.md
Original file line number Diff line number Diff line change
Expand Up @@ -43,7 +43,7 @@ The following examples demonstrate different use cases for using Kueue in Python
This example demonstrates installing Kueue to an existing cluster. You can save this
script to your local machine as `install-kueue-queues.py`.

{{% include "python/sample-job.py" "python" %}}
{{% include "python/install-kueue-queues.py" "python" %}}

And then run as follows:

Expand Down Expand Up @@ -98,3 +98,64 @@ Use:

You can also change the container image with `--image` and args with `--args`.
For more customization, you can edit the example script.

### Interact with Queues and Jobs

If you are developing an application that submits jobs and needs to interact
with and check on them, you likely want to interact with queues or jobs directly.
After running the example above, you can test the following example to interact
with the results. Write the following to a script called `sample-queue-control.py`.

{{% include "python/install-kueue-queues.py" "python" %}}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should this be sample-queue-control.py?

Is there a file called sample-queue-control.py?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yep - it got lost in the (now several) rebases but thankfully I had it in another place!


To make the output more interesting, we can run a few random jobs first:

```bash
python sample-job.py
python sample-job.py
python sample-job.py --job-name tacos
```

And then run the script to see your queue and sample job that you submit previously.

```bash
python sample-queue-control.py
```
```console
⛑️ Local Queues
Found queue user-queue
Admitted workloads: 3
Pending workloads: 0
Flavor default-flavor has resources [{'name': 'cpu', 'total': '3'}, {'name': 'memory', 'total': '600Mi'}]

💼️ Jobs
Found job sample-job-8n5sb
Succeeded: 3
Ready: 0
Found job sample-job-gnxtl
Succeeded: 1
Ready: 0
Found job tacos46bqw
Succeeded: 1
Ready: 1
```

If you wanted to filter jobs to a specific queue, you can do this via the job labels
under `job["metadata"]["labels"]["kueue.x-k8s.io/queue-name"]'. To list a specific job by
name, you can do:

```python
from kubernetes import client, config

# Interact with batch
config.load_kube_config()
batch_api = client.BatchV1Api()

# This is providing the name, and namespace
job = batch_api.read_namespaced_job("tacos46bqw", "default")
print(job)
```

See the [BatchV1](https://github.com/kubernetes-client/python/blob/master/kubernetes/docs/BatchV1Api.md)
API documentation for more calls.

20 changes: 16 additions & 4 deletions site/static/examples/python/install-kueue-queues.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@
import argparse

# install-kueue-queues.py will:
# 1. install queue from the latest on GitHub
# 1. install queue from the latest or a specific version on GitHub
# This example will demonstrate installing Kueue and applying a YAML file (local) to install Kueue

# Make sure your cluster is running!
Expand All @@ -22,8 +22,8 @@ def get_parser():
)
parser.add_argument(
"--version",
help="Version of Kueue to install",
default="0.4.0",
help="Version of Kueue to install (if undefined, will install from master branch)",
default=None,
)
return parser

Expand All @@ -39,12 +39,24 @@ def main():
install_kueue(args.version)


def get_install_url(version):
"""
Get the install version.

If a version is specified, use it. Otherwise install
from the main branch.
"""
if version is not None:
return f"https://github.com/kubernetes-sigs/kueue/releases/download/v{version}/manifests.yaml"
return "https://github.com/kubernetes-sigs/kueue/config/default?ref=main"


def install_kueue(version):
"""
Install Kueue of a particular version.
"""
print("⭐️ Installing Kueue...")
url = f"https://github.com/kubernetes-sigs/kueue/releases/download/v{version}/manifests.yaml"
url = get_install_url(version)
with tempfile.NamedTemporaryFile(delete=True) as install_yaml:
res = requests.get(url)
assert res.status_code == 200
Expand Down