Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): runtime parameters API for adding and using default parameters in protocols #14668

Merged
merged 20 commits into from
Mar 19, 2024
Merged
Show file tree
Hide file tree
Changes from 15 commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
31 changes: 31 additions & 0 deletions api/src/opentrons/protocol_api/_parameters.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,31 @@
from typing import Dict, Optional, Any

from opentrons.protocols.parameters.types import AllowedTypes, ParameterNameError


class Parameters:
def __init__(self, parameters: Optional[Dict[str, AllowedTypes]] = None) -> None:
self._values: Dict[str, AllowedTypes] = {}
if parameters is not None:
for name, value in parameters.items():
self._initialize_parameter(name, value)

def _getparam(self, variable_name: str) -> Any:
return getattr(self, f"_{variable_name}")

def _initialize_parameter(self, variable_name: str, value: AllowedTypes) -> None:
if not hasattr(self, variable_name) and not hasattr(self, f"_{variable_name}"):
setattr(self, f"_{variable_name}", value)
prop = property(
fget=lambda s, v=variable_name: Parameters._getparam(s, v) # type: ignore[misc]
)
setattr(Parameters, variable_name, prop)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A couple of suggestions for simplifying this.


First, instead of the user-facing API being like this:

def run(context):
    sample_count = context.params.sample_count

Have you considered making it this?

def run(context):
    sample_count = context.params["sample_count"]
  • This gives us namespacing in case we ever need to add more methods like params.get_all().
  • This matches parameter usage more closely to parameter declaration. In the declaration, the protocol author wrote variable_name="sample_count", with a string literal, so they pass the same string literal here.
  • This is arguably more straightforward and easier to explain.

Or, taking the context.params.sample_count syntax for granted, an implementation-level suggestion.

Right now, when we initialize the parameter foo="bar", we:

  1. Set a private attribute self._foo = "bar"
  2. Set a public attribute self.foo to a read-only property that returns self._foo
  3. Store "foo": "bar" in the self._values dict

Does this get easier if you treat self._values as the single source of truth, and override __getattr__() (and possibly __setattr__()) to delegate to it? In other words, instead of dynamically adding attributes, implement dynamic attribute lookup. Then you don't need the f"_{variable_name}" magic.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For now I'm going to leave the syntax the same, but I will take your suggestion to simplify things by using self._values as the source of truth for the fget function of the property.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this will be moot if you use self._values as the source of truth, but I've also just realized that in setattr(Parameters, variable_name, prop), it looks like that's mutating the Parameters class, which will leak between protocols?

self._values[variable_name] = value
else:
raise ParameterNameError(
f"Cannot use {variable_name} as a variable name, either duplicates another"
f" parameter name, Opentrons reserved function, or Python built-in"
)

def get_all(self) -> Dict[str, AllowedTypes]:
return self._values
169 changes: 169 additions & 0 deletions api/src/opentrons/protocol_api/parameter_context.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefix this file name with an underscore unless you want from opentrons.protocol_api import parameter_context to be a thing that protocol authors are allowed to do.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The would be no code execution reason to import the module, but since its the most public facing part of this (as far as the API) I'm alright keeping it without an underscore. Plus it would allow as user to do this pattern of typing the add_parameters function.

from opentrons.protocol_api.parameter_context import ParameterContext

def add_parameters(parameters: ParameterContext):
    ...

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would solve the type annotation problem by re-exporting the class so it’s

from opentrons.protocol_api import ParameterContext

Original file line number Diff line number Diff line change
@@ -0,0 +1,169 @@
"""Parameter context for python protocols."""

from typing import List, Optional, Union

from opentrons.protocols.api_support.types import APIVersion
from opentrons.protocols.parameters import parameter_definition
from opentrons.protocols.parameters.types import ParameterChoices

from ._parameters import Parameters

_ParameterDefinitionTypes = Union[
parameter_definition.ParameterDefinition[int],
parameter_definition.ParameterDefinition[bool],
parameter_definition.ParameterDefinition[float],
parameter_definition.ParameterDefinition[str],
]


class ParameterContext:
"""Public context for adding parameters to a protocol."""

def __init__(self, api_version: APIVersion) -> None:
"""Initializes a parameter context for user-set parameters."""
self._api_version = api_version
self._parameters: List[_ParameterDefinitionTypes] = []

def add_int(
self,
display_name: str,
variable_name: str,
default: int,
minimum: Optional[int] = None,
maximum: Optional[int] = None,
choices: Optional[List[ParameterChoices]] = None,
description: Optional[str] = None,
unit: Optional[str] = None,
) -> None:
"""Creates an integer parameter, settable within a given range or list of choices.

Arguments:
display_name: The display name of the int parameter as it would show up on the frontend.
variable_name: The variable name the int parameter will be referred to in the run context.
default: The default value the int parameter will be set to. This will be used in initial analysis.
minimum: The minimum value the int parameter can be set to (inclusive). Mutually exclusive with choices.
maximum: The maximum value the int parameter can be set to (inclusive). Mutually exclusive with choices.
choices: A list of possible choices that this parameter can be set to.
Mutually exclusive with minimum and maximum.
description: A description of the parameter as it will show up on the frontend.
unit: An optional unit to be appended to the end of the integer as it shown on the frontend.
"""
self._parameters.append(
parameter_definition.create_int_parameter(
display_name=display_name,
variable_name=variable_name,
default=default,
minimum=minimum,
maximum=maximum,
choices=choices,
description=description,
unit=unit,
)
)

def add_float(
self,
display_name: str,
variable_name: str,
default: float,
minimum: Optional[float] = None,
maximum: Optional[float] = None,
choices: Optional[List[ParameterChoices]] = None,
description: Optional[str] = None,
unit: Optional[str] = None,
) -> None:
"""Creates a float parameter, settable within a given range or list of choices.

Arguments:
display_name: The display name of the float parameter as it would show up on the frontend.
variable_name: The variable name the float parameter will be referred to in the run context.
default: The default value the float parameter will be set to. This will be used in initial analysis.
minimum: The minimum value the float parameter can be set to (inclusive). Mutually exclusive with choices.
maximum: The maximum value the float parameter can be set to (inclusive). Mutually exclusive with choices.
choices: A list of possible choices that this parameter can be set to.
Mutually exclusive with minimum and maximum.
description: A description of the parameter as it will show up on the frontend.
unit: An optional unit to be appended to the end of the float as it shown on the frontend.
"""
self._parameters.append(
parameter_definition.create_float_parameter(
display_name=display_name,
variable_name=variable_name,
default=default,
minimum=minimum,
maximum=maximum,
choices=choices,
description=description,
unit=unit,
)
)

def add_bool(
self,
display_name: str,
variable_name: str,
default: bool,
description: Optional[str] = None,
) -> None:
"""Creates a boolean parameter with allowable values of "On" (True) or "Off" (False).

Arguments:
display_name: The display name of the boolean parameter as it would show up on the frontend.
variable_name: The variable name the boolean parameter will be referred to in the run context.
default: The default value the boolean parameter will be set to. This will be used in initial analysis.
description: A description of the parameter as it will show up on the frontend.
"""
self._parameters.append(
parameter_definition.create_bool_parameter(
display_name=display_name,
variable_name=variable_name,
default=default,
choices=[
{"display_name": "On", "value": True},
{"display_name": "Off", "value": False},
],
description=description,
)
)

def add_str(
self,
display_name: str,
variable_name: str,
default: str,
choices: Optional[List[ParameterChoices]] = None,
description: Optional[str] = None,
) -> None:
"""Creates a string parameter, settable among given choices.

Arguments:
display_name: The display name of the string parameter as it would show up on the frontend.
variable_name: The variable name the string parameter will be referred to in the run context.
default: The default value the string parameter will be set to. This will be used in initial analysis.
choices: A list of possible choices that this parameter can be set to.
Mutually exclusive with minimum and maximum.
description: A description of the parameter as it will show up on the frontend.
"""
self._parameters.append(
parameter_definition.create_str_parameter(
display_name=display_name,
variable_name=variable_name,
default=default,
choices=choices,
description=description,
)
)

def export_parameters(self) -> Parameters:
"""Exports all parameters into a protocol run usable parameters object.

:meta private:

This is intended for Opentrons internal use only and is not a guaranteed API.
"""
return Parameters(
parameters={
parameter.variable_name: parameter.value
for parameter in self._parameters
}
)
6 changes: 6 additions & 0 deletions api/src/opentrons/protocol_api/protocol_context.py
Original file line number Diff line number Diff line change
Expand Up @@ -64,6 +64,7 @@
MagneticBlockContext,
ModuleContext,
)
from ._parameters import Parameters


logger = logging.getLogger(__name__)
Expand Down Expand Up @@ -167,6 +168,7 @@ def __init__(
self._core.load_ot2_fixed_trash_bin()

self._commands: List[str] = []
self._params: Parameters = Parameters()
self._unsubscribe_commands: Optional[Callable[[], None]] = None
self.clear_commands()

Expand Down Expand Up @@ -215,6 +217,10 @@ def bundled_data(self) -> Dict[str, bytes]:
"""
return self._bundled_data

@property
def params(self) -> Parameters:
return self._params
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might want a @requires_version decorator or a ticket for it, before we forget.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can create a feature flag for now and put it behind that. Replace it with API level check before release. The feature flag will be useful for other backend things too.


def cleanup(self) -> None:
"""Finalize and clean up the protocol context."""
if self._unsubscribe_commands:
Expand Down
52 changes: 45 additions & 7 deletions api/src/opentrons/protocols/execution/execute_python.py
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,12 @@

from opentrons.drivers.smoothie_drivers.errors import SmoothieAlarm
from opentrons.protocol_api import ProtocolContext
from opentrons.protocol_api._parameters import Parameters
from opentrons.protocol_api.parameter_context import ParameterContext
from opentrons.protocols.execution.errors import ExceptionInProtocolError
from opentrons.protocols.types import PythonProtocol, MalformedPythonProtocolError


from opentrons_shared_data.errors.exceptions import ExecutionCancelledError

MODULE_LOG = logging.getLogger(__name__)
Expand All @@ -29,6 +33,14 @@ def _runfunc_ok(run_func: Any):
)


def _add_parameters_func_ok(add_parameters_func: Any) -> None:
if not callable(add_parameters_func):
raise SyntaxError("'add_parameters' must be a function.")
sig = inspect.Signature.from_callable(add_parameters_func)
if len(sig.parameters) != 1:
raise SyntaxError("Function 'add_parameters' must take exactly one argument.")


def _find_protocol_error(tb, proto_name):
"""Return the FrameInfo for the lowest frame in the traceback from the
protocol.
Expand All @@ -41,6 +53,34 @@ def _find_protocol_error(tb, proto_name):
raise KeyError


def _raise_pretty_protocol_error(exception: Exception, filename: str) -> None:
exc_type, exc_value, tb = sys.exc_info()
try:
frame = _find_protocol_error(tb, filename)
except KeyError:
# No pretty names, just raise it
raise exception
raise ExceptionInProtocolError(
exception, tb, str(exception), frame.lineno
) from exception


def _parse_and_set_parameters(
protocol: PythonProtocol, new_globs: Dict[Any, Any], filename: str
) -> Parameters:
try:
_add_parameters_func_ok(new_globs.get("add_parameters"))
except SyntaxError as se:
raise MalformedPythonProtocolError(str(se))
parameter_context = ParameterContext(api_version=protocol.api_level)
new_globs["__param_context"] = parameter_context
try:
exec("add_parameters(__param_context)", new_globs)
except Exception as e:
_raise_pretty_protocol_error(exception=e, filename=filename)
return parameter_context.export_parameters()


def run_python(proto: PythonProtocol, context: ProtocolContext):
new_globs: Dict[Any, Any] = {}
exec(proto.contents, new_globs)
Expand All @@ -60,10 +100,14 @@ def run_python(proto: PythonProtocol, context: ProtocolContext):
# AST filename.
filename = proto.filename or "<protocol>"

if new_globs.get("add_parameters"):
context._params = _parse_and_set_parameters(proto, new_globs, filename)

try:
_runfunc_ok(new_globs.get("run"))
except SyntaxError as se:
raise MalformedPythonProtocolError(str(se))

new_globs["__context"] = context
try:
exec("run(__context)", new_globs)
Expand All @@ -75,10 +119,4 @@ def run_python(proto: PythonProtocol, context: ProtocolContext):
# this is a protocol cancel and shouldn't have special logging
raise
except Exception as e:
exc_type, exc_value, tb = sys.exc_info()
try:
frame = _find_protocol_error(tb, filename)
except KeyError:
# No pretty names, just raise it
raise e
raise ExceptionInProtocolError(e, tb, str(e), frame.lineno) from e
_raise_pretty_protocol_error(exception=e, filename=filename)
Empty file.
Loading
Loading