Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(api): proper LiquidHandlingPropertyByVolume class and validation for setting liquid class properties #16725

Merged
merged 11 commits into from
Nov 12, 2024

Conversation

jbleon95
Copy link
Contributor

@jbleon95 jbleon95 commented Nov 7, 2024

Overview

Closes AUTH-837

This PR adds a proper LiquidHandlingPropertyByVolume class, replacing the previous dictionary of string keys to float values that was the literal representation of the liquid class schema. Now it uses that dictionary to make a default value as well as functions to take any volume and interpolate the proper volume based on that. Methods are also there for setting new points for volume and deleting existing points.

The other part of this PR is adding validation for the different settable values in liquid class that would require validating before hand. This includes boolean values, floating points, positive ints and floats, and xyz coordinates.

Changelog

  • added LiquidHandlingPropertyByVolume class to handle interpolation for these properties and adding/deleting points
  • added validators for all settable liquid class properties that need them for clear errors

Review requests

Should all the new validators live in the general PAPI validation.py class or should they exist elsewhere?

Risk assessment

Low.

@jbleon95 jbleon95 requested review from sanni-t and ddcc4 November 7, 2024 19:34
@jbleon95 jbleon95 requested a review from a team as a code owner November 7, 2024 19:34
Copy link

codecov bot commented Nov 7, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 92.43%. Comparing base (094be6e) to head (91ce7c8).
Report is 81 commits behind head on edge.

Additional details and impacted files

Impacted file tree graph

@@           Coverage Diff           @@
##             edge   #16725   +/-   ##
=======================================
  Coverage   92.43%   92.43%           
=======================================
  Files          77       77           
  Lines        1283     1283           
=======================================
  Hits         1186     1186           
  Misses         97       97           
Flag Coverage Δ
g-code-testing 92.43% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Copy link
Member

@sanni-t sanni-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks mostly good! Just a few suggestions. The only thing that sticks out to me is the properties_by_volume property.

# numpy interp does not automatically sort the points used for interpolation, so
# here we are sorting them by the keys (volume) and then using zip to get two
# tuples in the correct order
sorted_volumes, sorted_values = zip(*sorted(self._properties_by_volume.items()))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if we we make a separate function for fetching the sorted dict and add caching to it so that this function doesn't end up doing sorts again and again on the same set of data? Slight efficiency improvement

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just store the raw values sorted? Do we really need to preserve the original unsorted order that the user gave us? (But also, I'd expect the properties are so small -- like 5 entries -- that the sort time is negligible compared to everything else you're doing.)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'll take a combination of yours and David's suggestion and just store these values in the class and update them whenever a value is added/deleted. It's functionally identical to caching it since that value would need to be updated anyway anytime a value is added/removed.

# here we are sorting them by the keys (volume) and then using zip to get two
# tuples in the correct order
sorted_volumes, sorted_values = zip(*sorted(self._properties_by_volume.items()))
return float(interp(validated_volume, sorted_volumes, sorted_values))
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, not sure if the interpolator goes through the trouble of actually doing the interpolation even for values that are readily available in the dictionary. If it does, then I think it'll be a bit better to first check if the volume exists in the dictionary and interpolate only if it doesn't exist.

api/src/opentrons/protocol_api/validation.py Outdated Show resolved Hide resolved
api/tests/opentrons/protocol_api/test_validation.py Outdated Show resolved Hide resolved
api/tests/opentrons/protocol_api/test_validation.py Outdated Show resolved Hide resolved
api/tests/opentrons/protocol_api/test_validation.py Outdated Show resolved Hide resolved
return self._default

@property
def properties_by_volume(self) -> Dict[Union[float, str], float]:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm.. I wonder if there's a more intuitive way of representing this. Ideally would be good to avoid a call like aspirate.flow_rate_by_volume.properties_by_volume.
I guess maybe calling this property/ function something like get_dict() might also be an option

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about calling this something like raw_properties(), to distinguish it from get_for_volume() below?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think I'll go with as_dict for now, can always update/change it until release.

return self._default

@property
def properties_by_volume(self) -> Dict[Union[float, str], float]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How about calling this something like raw_properties(), to distinguish it from get_for_volume() below?

# numpy interp does not automatically sort the points used for interpolation, so
# here we are sorting them by the keys (volume) and then using zip to get two
# tuples in the correct order
sorted_volumes, sorted_values = zip(*sorted(self._properties_by_volume.items()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we just store the raw values sorted? Do we really need to preserve the original unsorted order that the user gave us? (But also, I'd expect the properties are so small -- like 5 entries -- that the sort time is negligible compared to everything else you're doing.)

props_by_volume_copy["default"] = self._default
return props_by_volume_copy

def get_for_volume(self, volume: float) -> float:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

How does the default play into this function?

Also, for someone not so familiar with numpy, and not so familiar with the expected behavior of the machines, it'd be helpful to have a comment that just outright states that you're trying to do a piecewise linear interpolation.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I'm still confused about the behavior of default. Let's say someone gives you this config without any non-default points:

{"default": 5}

Is get_for_volume() expected to return 5?

return float(interp(validated_volume, sorted_volumes, sorted_values))

def set_for_volume(self, volume: float, value: float) -> None:
validated_volume = validation.ensure_positive_float(volume)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

FYI, forcing the control points on the interpolation curve to be strictly positive makes life annoying for users. Like, if I want to configure a rate of 3 per uL, the easiest way to do that would be:

{
0 uL: 0.0,
1000 uL: 3000.0 
}

But if you reject 0 uL because it's not a positive float, then it's harder to set up the curve.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

0 is an allowed value for this, I can rename this validation function if need be since I know zero isn't a positive number.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can rename this validation function if need be

I would prefer that, but otherwise this PR looks fine. I can't think of a good short name for "ensure positive or zero" though.

I will note that in the future, there will probably be things that we need to be strictly positive (flow rates, dimensions) and some things that can be positive-or-zero (like the control points for interpolation). If you claim the ensure_positive_float() name now, someone who wants to implement "ensure strictly positive float" later will have to work around your naming.

@jbleon95 jbleon95 requested review from sanni-t and ddcc4 November 8, 2024 19:21
api/src/opentrons/protocol_api/_liquid_properties.py Outdated Show resolved Hide resolved
props_by_volume_copy["default"] = self._default
return props_by_volume_copy

def get_for_volume(self, volume: float) -> float:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hm, I'm still confused about the behavior of default. Let's say someone gives you this config without any non-default points:

{"default": 5}

Is get_for_volume() expected to return 5?

api/src/opentrons/protocol_api/_liquid_properties.py Outdated Show resolved Hide resolved
Copy link
Member

@sanni-t sanni-t left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes look good! Thanks!

@jbleon95 jbleon95 merged commit ad0de84 into edge Nov 12, 2024
51 checks passed
@jbleon95 jbleon95 deleted the by_volume_liquid_class_and_validators branch November 12, 2024 18:21
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants