Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Function description from docstring #1700

Merged
merged 1 commit into from
Nov 14, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
New Functionality
^^^^^^^^^^^^^^^^^

- Function docstrings are now read and used as the description for the function when it
is uploaded. This will support future UI changes to the webapp.
chris-janidlo marked this conversation as resolved.
Show resolved Hide resolved
5 changes: 4 additions & 1 deletion compute_sdk/globus_compute_sdk/sdk/client.py
Original file line number Diff line number Diff line change
Expand Up @@ -591,7 +591,10 @@ def register_function(
container_uuid : str
Container UUID from registration with Globus Compute
description : str
Description of the file
Description of the function. If this is None, and the function has a
docstring, that docstring is uploaded as the function's description instead;
otherwise, if this has a value, it's uploaded as the description, even if
the function has a docstring.
metadata : dict
Function metadata (E.g., Python version used when serializing the function)
public : bool
Expand Down
3 changes: 3 additions & 0 deletions compute_sdk/globus_compute_sdk/sdk/web_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
`FunctionRegistrationData` which can be constructed from an arbitrary callable.
"""

import inspect
import json
import typing as t
import warnings
Expand Down Expand Up @@ -62,6 +63,8 @@ def __init__(
)
function_name = function.__name__
function_code = _get_packed_code(function, serializer=serializer)
if description is None:
description = inspect.getdoc(function)

chris-janidlo marked this conversation as resolved.
Show resolved Hide resolved
if function_name is None or function_code is None:
raise ValueError(
Expand Down
72 changes: 72 additions & 0 deletions compute_sdk/tests/unit/test_client.py
Original file line number Diff line number Diff line change
@@ -1,3 +1,4 @@
import inspect
import sys
import uuid
from unittest import mock
Expand Down Expand Up @@ -331,6 +332,77 @@ def test_register_function_deprecated_args(gcc, dep_arg):
assert dep_arg in str(warning)


def _docstring_test_case_no_docstring():
pass


def _docstring_test_case_single_line():
"""This is a docstring"""


def _docstring_test_case_multi_line():
"""This is a docstring
that spans multiple lines
and those lines are indented
"""


def _docstring_test_case_real_world():
"""
Register a task function with this Executor's cache.
All function execution submissions (i.e., ``.submit()``) communicate which
pre-registered function to execute on the endpoint by the function's
identifier, the ``function_id``. This method makes the appropriate API
call to the Globus Compute web services to first register the task function, and
then stores the returned ``function_id`` in the Executor's cache.
In the standard workflow, ``.submit()`` will automatically handle invoking
this method, so the common use-case will not need to use this method.
However, some advanced use-cases may need to fine-tune the registration
of a function and so may manually set the registration arguments via this
method.
If a function has already been registered (perhaps in a previous
iteration), the upstream API call may be avoided by specifying the known
``function_id``.
If a function already exists in the Executor's cache, this method will
raise a ValueError to help track down the errant double registration
attempt.
:param fn: function to be registered for remote execution
:param function_id: if specified, associate the ``function_id`` to the
``fn`` immediately, short-circuiting the upstream registration call.
:param func_register_kwargs: all other keyword arguments are passed to
the ``Client.register_function()``.
:returns: the function's ``function_id`` string, as returned by
registration upstream
:raises ValueError: raised if a function has already been registered with
this Executor
"""


@pytest.mark.parametrize(
"func",
[
_docstring_test_case_no_docstring,
_docstring_test_case_single_line,
_docstring_test_case_multi_line,
_docstring_test_case_real_world,
],
)
def test_register_function_docstring(gcc, func):
gcc.web_client = mock.MagicMock()

gcc.register_function(func)
expected = inspect.getdoc(func)

a, _ = gcc.web_client.register_function.call_args
func_data = a[0]
assert func_data.description == expected


def test_register_function_no_metadata(gcc):
gcc.web_client = mock.MagicMock()

Expand Down
31 changes: 29 additions & 2 deletions docs/sdk.rst
Original file line number Diff line number Diff line change
Expand Up @@ -51,6 +51,8 @@ is defined in the same way as any Python function before being registered with G
.. code-block:: python
def platform_func():
"""Get platform information about this system."""
import platform
return platform.platform()
Expand Down Expand Up @@ -106,6 +108,8 @@ The following example shows how strings can be passed to and from a function.
.. code-block:: python
def hello(firstname, lastname):
"""Say hello to someone."""
return 'Hello {} {}'.format(firstname, lastname)
func_id = gcc.register_function(hello)
Expand All @@ -127,7 +131,7 @@ To share with a group, set ``group=<globus_group_id>`` when registering a functi

.. code-block:: python
gcc.register_function(func, description="My function", group=<globus_group_id>)
gcc.register_function(func, group=<globus_group_id>)
Upon execution, Globus Compute will check group membership to ensure that the user is authorized to execute the function.
Expand All @@ -136,7 +140,28 @@ You can also set a function to be publicly accessible by setting ``public=True``

.. code-block:: python
gcc.register_function(func, description="My function", public=True)
gcc.register_function(func, public=True)
To add a description to a function, you can either set ``description=<my_description>``
when calling ``register_function``, or add a docstring to the function. Note that the
latter also works with the ``Executor`` class.

.. code-block:: python
gcc.register_function(func, description="My function")
def function_with_docstring():
"""My function, with a docstring"""
return "foo"
gcc.register_function(func) # description is automatically read from the docstring
gcx = Executor()
fut = gcx.submit(function_with_docstring) # automatically registers the function with its docstring
# if both are specified, the argument wins
gcc.register_function(function_with_docstring, description="this has priority over docstrings")
.. _batching:
Expand Down Expand Up @@ -307,6 +332,8 @@ method:
from globus_compute_sdk.serialize import ComputeSerializer, DillCodeSource, JSONData
def greet(name, greeting = "greetings"):
"""Greet someone."""
return f"{greeting} {name}"
serializer = ComputeSerializer(
Expand Down
Loading