Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

docs: add a python sdk example for log following. #8981

Merged
merged 1 commit into from
Apr 8, 2024
Merged

docs: add a python sdk example for log following. #8981

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
75 changes: 43 additions & 32 deletions docs/reference/python-sdk/example-workflows.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,56 +6,67 @@

Walk through how to use the Python SDK in these basic and advanced workflow examples.

.. tabs::
************************************
Find the Top Performing Checkpoint
************************************

.. tab::
In this example, we'll walk through the most basic workflow for creating an experiment, waiting for
it to complete, and finding the top-performing checkpoint.

Basic Workflow
The first step is to import the client module and possibly to call login():

**Find the Top Performing Checkpoint**
.. code:: python

In this example, we'll walk through the most basic workflow for creating an experiment,
waiting for it to complete, and finding the top-performing checkpoint.
from determined.experimental import client

The first step is to import the client module and possibly to call login():
# We will assume that you have called `det user login`, so this is unnecessary:
# client.login(master=..., user=..., password=...)

.. code:: python
The next step is to call create_experiment():

from determined.experimental import client
.. code:: python

# We will assume that you have called `det user login`, so this is unnecessary:
# client.login(master=..., user=..., password=...)
# Config can be a path to a config file or a Python dict of the config.
exp = client.create_experiment(config="my_config.yaml", model_dir=".")
print(f"started experiment {exp.id}")

The next step is to call create_experiment():
The returned object is an ``Experiment`` object, which offers methods to manage the experiment's
lifecycle. In the following example, we simply await the experiment's completion.

.. code:: python
.. code:: python

# Config can be a path to a config file or a Python dict of the config.
exp = client.create_experiment(config="my_config.yaml", model_dir=".")
print(f"started experiment {exp.id}")
exit_status = exp.wait()
print(f"experiment completed with status {exit_status}")

The returned object is an ``Experiment`` object, which offers methods to manage the
experiment's lifecycle. In the following example, we simply await the experiment's completion.
Now that the experiment has completed, you can grab the top-performing checkpoint from training:

.. code:: python
.. code:: python

exit_status = exp.wait()
print(f"experiment completed with status {exit_status}")
best_checkpoint = exp.list_checkpoints()[0]
print(f"best checkpoint was {best_checkpoint.uuid}")

Now that the experiment has completed, you can grab the top-performing checkpoint from
training:
******************************************
Create an Experiment and Follow its Logs
******************************************

.. code:: python
Using ``det`` CLI, you can create an experiment and print its logs until completion using:

best_checkpoint = exp.list_checkpoints()[0]
print(f"best checkpoint was {best_checkpoint.uuid}")
.. code:: bash

.. tab::
det e create --follow ...

Advanced Workflow
The same behavior can be replicated with the Python SDK:

**Run and Administer Experiments**
.. code:: python

Visit the `det-python-sdk-demo
<https://github.com/determined-ai/determined-examples/tree/e499000d92a0a973d1f40a419934f393957a3296/blog/python_sdk_demo>`__
to learn how to run and administer experiments using the Python SDK.
exp = client.create_experiment(...)
for logline in exp.await_first_trial().iter_logs():
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it obvious to the reader that this blocks on the production of log lines? If yes, cool. If not, I think that aspect deserves a small comment.

print(logline)

********************************
Run and Administer Experiments
********************************

Visit the `det-python-sdk-demo
<https://github.com/determined-ai/determined-examples/tree/e499000d92a0a973d1f40a419934f393957a3296/blog/python_sdk_demo>`__
to learn how to run and administer experiments using the Python SDK.
Loading