Skip to content

Commit

Permalink
feat(hardware-testing): grav script on ot3 (#12302)
Browse files Browse the repository at this point in the history
* moves gravimetric functionality into its own subfolder

* move all ot2 functionality into gravimetric project

* wip

* change tipracks to ot3 versions

* can simulate on OT3

* runs on ot3 without error

* removes ProtocolContext from scale implementation

* push-plot-webpage-ot3 works

* wip

* DELETE THIS COMMIT

* adds DVT pipettes to shared data

* ot3simulator small to change to allow simulation

* local simulation working

* use passed loop from ThreadManager when build OT3APIg

* use correct filepath for OT3 data files

* move execute to single file

* placeholder for liquid-class defaults for OT3

* only read from scale when needed

* remove pipette-timestamps

* integrates with scale; adds more volumes

* wip

* remove hack from before revisioning update

* remove unnecessary v3.4 pipettes from shared-data

* redo default liquid class settings for OT3 qc volumes

* deletes a lot of stuff

* adds comments on reduced leading-air-gaps

* interpolate between liquid classes; move labware-def to static python file

* increments test with cleared pipette ul-per-mm

* get labware-offsets; low_volume flag in config to filter <2uL

* linting

* adds is_simulator to scale; reduce 50ul leading air-gaps to avoid SW bug

* adds blank measurements to find evaporation rate

* add mix before aspirate

* remove aspirate after blow-out

* change wording evaporation to blank

* adds blowout with microliters to HW api

* use HW api to run blow-out with microliters within grav script

* reorganizes pipetting sequence code; adds verbose comments

* wip: adding test report

* wip: test-report is generated and format seems correct

* wip: most data stored, just need to calculate volumes

* wip: more cleaning up; store trial volumes

* entire test report is filled out

* calculate volumes using grams and environment data

* adds more args; adds fake protocol for calibrating tipracks

* updates vial labware def

* don't fail out if the rear-panel isn't found since EVT bots don't have one yet

* put usb connected rear-panel behind a feature flag

* update robot server tests

* few fixups from rebase

* linting error from pipette qc script

* slowing to 50mm/sec reduces noise on scale from moving air

* use encoders after pick-up-tip

* adds return tip option; multiple tip-racks

* add hard-coded vial offset to get around App bug

* adds script for finding labware offset

* option to skip blank readings

* correct decimal place on uL calculations

* keep tip farther away from liquid during blank measurements

* also limit dispense volumes to 90% of pipette max

* print final results at end

* fix(engine): subtract nominal overlap when calculating nominal tip le

* slow down acceleration/discontinuity; hard-code homing speeds to be faster

* testing went well

* linting

* adds inspect and skip-mix arguments

* better printout during test run

* removing warnings about not having tip-length calibration on OT3

* adds starting-tip

* use same submerge/retract distances for aspirate and dispense

* adds pipette temperature readings

* --inspect speeds all measurements up

* use all of plunger

* test more options in makefile

* increase p1000-single pick-up-current from 0.15 to 0.25

* fix bug where 1x trial would break CV calculations

* wip: tuning mix so droplets aren't created

* increase T50 leading-air-gaps

* update liquid-class settings to match tested values from Nick

* adds --user-volumes

* raise top plunger positions from 0.5 to 0.0

* reduce amount of logs during measurement delay

* load latest data 100ms after finishing previous data

* different speeds for retract/submerge

* combine all stable samples into one segment to average; reduce delay time to 10 seconds

* reduct p1000 leading-air-gap from 32 to 16, so only one bubble is ejected while submerged

* change skip-mix and skip-blank args to mix and blank

* linting!

* remove waiting for user to confirmware labware offsets

* removes unnecessary changes to api

* remove reading temp-humidity within script

* remove blow-out with volume from api

* plate blow-out with micro-liters in hardware-testing script

* linting!

* remove shared-data changes

* delete temp script for testing labware offsets

* remove tip-overlap bug fix

* fix broken test

* fix makefile incorrectly calling gravimetric script

* json uses 2-space indentation

* format-js

---------

Co-authored-by: Ryan howard <[email protected]>
  • Loading branch information
andySigler and ryanthecoder authored Mar 17, 2023
1 parent 4850e2b commit 055fafa
Show file tree
Hide file tree
Showing 65 changed files with 2,482 additions and 19,261 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -377,7 +377,7 @@ def _attached_pipette_to_mount(
raise RuntimeError(
f"mount {mount.name} requested a {expected_instr} which is not supported on the OT3"
)
if found_model and expected_instr and (expected_instr != found_model):
if found_model and expected_instr and (expected_instr not in found_model):
if self._strict_attached:
raise RuntimeError(
"mount {}: expected instrument {} but got {}".format(
Expand Down
60 changes: 32 additions & 28 deletions hardware-testing/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -71,43 +71,55 @@ test:
test-cov:
$(pytest) $(tests) $(test_opts) $(cov_opts)

.PHONY: test-integration
test-integration:
.PHONY: test-gravimetric
test-gravimetric:
$(python) -m hardware_testing.gravimetric --simulate --pipette 1000 --tip 1000 --trials 3 --starting-tip A1
$(python) -m hardware_testing.gravimetric --simulate --pipette 1000 --tip 200 --trials 3 --starting-tip A1
$(python) -m hardware_testing.gravimetric --simulate --pipette 1000 --tip 50 --trials 3 --starting-tip A1
$(python) -m hardware_testing.gravimetric --simulate --pipette 50 --tip 50 --trials 3 --starting-tip A1
$(python) -m hardware_testing.gravimetric --simulate --pipette 1000 --tip 1000 --trials 3 --starting-tip A1 --inspect
$(python) -m hardware_testing.gravimetric --simulate --pipette 1000 --tip 1000 --trials 3 --starting-tip A1 --increment

.PHONY: test-production-qc
test-production-qc:
$(python) -m hardware_testing.production_qc.pipette_assembly_qc_ot3 --operator test --simulate
$(python) -m hardware_testing.production_qc.robot_assembly_qc_ot3 --simulate
$(python) protocols/ot2_p300_single_channel_gravimetric.py --simulate
$(python) -m hardware_testing.examples.test_report

.PHONY: test-examples
test-examples:
$(python) -m hardware_testing.examples.csv_report
$(python) -m hardware_testing.examples.custom_axis_settings_ot3 --simulate
$(python) -m hardware_testing.examples.endstop_encoder_ot3 --simulate
$(python) -m hardware_testing.examples.pick_up_tip_ot3 --simulate
$(python) -m hardware_testing.examples.plunger_ot3 --simulate
$(python) -m hardware_testing.examples.capacitive_probe_ot3 --simulate

.PHONY: test-scripts
test-scripts:
$(python) -m hardware_testing.scripts.bowtie_ot3 --simulate

.PHONY: test-integration
test-integration: test-gravimetric test-production-qc test-examples test-scripts

.PHONY: lint
lint:
$(python) -m mypy hardware_testing tests protocols
$(python) -m black --check hardware_testing tests protocols setup.py
$(python) -m flake8 hardware_testing tests protocols setup.py
$(python) -m mypy hardware_testing tests
$(python) -m black --check hardware_testing tests setup.py
$(python) -m flake8 hardware_testing tests setup.py

.PHONY: format
format:
$(python) -m black hardware_testing tests protocols setup.py
$(python) -m black hardware_testing tests setup.py

define move-plot-webpage
define move-plot-webpage-ot3
ssh -i $(2) $(3) root@$(1) \
"function cleanup () { mount -o remount,ro / ; } ;\
mount -o remount,rw / &&\
mv /data/plot/index.html /usr/lib/python3.7/site-packages/hardware_testing/tools/plot &&\
mv /data/plot/index.js /usr/lib/python3.7/site-packages/hardware_testing/tools/plot &&\
mv /data/plot/index.html /opt/opentrons-robot-server/hardware_testing/tools/plot &&\
mv /data/plot/index.js /opt/opentrons-robot-server/hardware_testing/tools/plot &&\
rm -rf /data/plot &&\
cleanup || cleanup"
endef

.PHONY: push-plot-webpage
push-plot-webpage:
scp -r hardware_testing/tools/plot root@$(host):/data
$(call move-plot-webpage,$(host),$(br_ssh_key),$(ssh_opts))

.PHONY: push-plot-webpage-ot3
push-plot-webpage-ot3:
scp -r hardware_testing/tools/plot root@$(host):/data
Expand Down Expand Up @@ -135,19 +147,11 @@ push-no-restart-ot3: sdist Pipfile.lock
.PHONY: push-ot3
push-ot3: push-no-restart-ot3 restart-ot3

.PHONY: push-protocols
push-protocols:
scp -r protocols root@$(host):/data/user_storage/opentrons_data

.PHONY: push-protocols-ot3
push-protocols-ot3:
scp -r protocols root@$(host):/opt/opentrons-robot-server

.PHONY: push-all
push-all: clean wheel push-no-restart push-plot-webpage push-protocols
push-all: clean wheel push-no-restart push-plot-webpage

.PHONY: push-all-ot3
push-all: clean wheel push-no-restart-ot3 push-plot-webpage-ot3 push-protocols-ot3
push-all-ot3: push-ot3 push-plot-webpage-ot3

.PHONY: term
term:
Expand Down Expand Up @@ -197,7 +201,7 @@ sync-sw-ot3:
cd ../hardware && $(MAKE) push-no-restart-ot3 host=$(host)
cd ../api && $(MAKE) push-no-restart-ot3 host=$(host)
cd ../shared-data && $(MAKE) all push-no-restart-ot3 host=$(host)
cd ../hardware-testing && $(MAKE) push-no-restart-ot3 host=$(host)
cd ../hardware-testing && $(MAKE) push-all-ot3 host=$(host)

.PHONY: sync-fw-ot3
sync-fw-ot3:
Expand Down
36 changes: 26 additions & 10 deletions hardware-testing/hardware_testing/data/csv_report.py
Original file line number Diff line number Diff line change
Expand Up @@ -69,6 +69,11 @@ def __str__(self) -> str:
_elapsed = round(self._elapsed_time, 1)
return f"{_elapsed},{full_str}"

@property
def data(self) -> List[Any]:
"""Data."""
return self._data

@property
def tag(self) -> str:
"""Line tag."""
Expand Down Expand Up @@ -268,18 +273,23 @@ def _generate_results_overview_section(tags: List[str]) -> CSVSection:
class CSVReport:
"""CSV Report."""

def __init__(self, script_path: str, sections: List[CSVSection]) -> None:
def __init__(
self,
test_name: str,
sections: List[CSVSection],
run_id: Optional[str] = None,
start_time: Optional[float] = None,
) -> None:
"""CSV Report init."""
self._script_path = script_path
self._test_name = data_io.create_test_name_from_file(script_path)
self._run_id = data_io.create_run_id()
self._test_name = test_name
self._run_id = run_id if run_id else data_io.create_run_id()
self._tag: Optional[str] = None
self._file_name: Optional[str] = None
_section_meta = _generate_meta_data_section()
_section_titles = [s.title for s in sections]
_section_results = _generate_results_overview_section(_section_titles)
self._sections = [_section_meta, _section_results] + sections
self._cache_start_time() # must happen before storing any data
self._cache_start_time(start_time) # must happen before storing any data
self(META_DATA_TITLE, META_DATA_TEST_NAME, [self._test_name])
self(META_DATA_TITLE, META_DATA_TEST_RUN_ID, [self._run_id])
_now = datetime.utcnow().strftime("%Y/%m/%d-%H:%M:%S")
Expand Down Expand Up @@ -343,23 +353,29 @@ def parent(self) -> Path:
"""Parent directory of this report file."""
return data_io.create_folder_for_test_data(self._test_name)

def _cache_start_time(self) -> None:
start_time = time()
@property
def tag(self) -> str:
"""Tag."""
return f"{self.__class__.__name__}-{self._tag}"

def _cache_start_time(self, start_time: Optional[float] = None) -> None:
checked_start_time = start_time if start_time else time()
for section in self._sections:
for line in section.lines:
if isinstance(line, CSVLineRepeating):
for i in range(len(line)):
line[i].cache_start_time(start_time)
line[i].cache_start_time(checked_start_time)
else:
line.cache_start_time(start_time)
line.cache_start_time(checked_start_time)

def set_tag(self, tag: str) -> None:
"""CSV Report set tag."""
self._tag = tag
self(META_DATA_TITLE, META_DATA_TEST_TAG, [self._tag])
self._file_name = data_io.create_file_name(
self._test_name, self._run_id, self._tag
self._test_name, self._run_id, self.tag
)
self.save_to_disk()

def set_operator(self, operator: str) -> None:
"""Set operator."""
Expand Down
16 changes: 14 additions & 2 deletions hardware-testing/hardware_testing/drivers/radwag/driver.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,6 @@
from abc import ABC, abstractmethod
from typing import Tuple, Optional

from random import uniform
from serial import Serial # type: ignore[import]

from .commands import (
Expand Down Expand Up @@ -226,6 +225,15 @@ def read_mass(self) -> Tuple[float, bool]:
class SimRadwagScale(RadwagScaleBase):
"""Simulating Radwag Scale Driver."""

def __init__(self) -> None:
"""Constructor."""
self._mass: float = 0.0

@property
def sim_mass(self) -> float:
"""Simulation mass."""
return self._mass

def connect(self) -> None:
"""Connect."""
return
Expand Down Expand Up @@ -268,4 +276,8 @@ def set_tare(self, tare: float) -> None:

def read_mass(self) -> Tuple[float, bool]:
"""Read mass."""
return uniform(2.5, 2), True
return self._mass, True

def set_simulation_mass(self, mass: float) -> None:
"""Set simulation mass."""
self._mass = mass
Original file line number Diff line number Diff line change
Expand Up @@ -10,15 +10,8 @@

NUM_REPEATING_DATA = 30
EXAMPLE_REPORT = CSVReport(
script_path=__file__,
test_name="example-test-report",
sections=[
CSVSection(
title="TEST-INFO",
lines=[
CSVLine("software-version", [str]),
CSVLine("operator", [str]),
],
),
CSVSection(
title="PIPETTE-LEFT",
lines=[
Expand Down
39 changes: 39 additions & 0 deletions hardware-testing/hardware_testing/examples/pipette_sensors_ot3.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,39 @@
"""Pipette sensors OT3."""
import argparse
import asyncio
from time import sleep

from hardware_testing.opentrons_api import helpers_ot3
from hardware_testing.opentrons_api.types import OT3Mount


async def _main(is_simulating: bool) -> None:
api = await helpers_ot3.build_async_ot3_hardware_api(
is_simulating=is_simulating,
pipette_left="p1000_single_v3.3",
pipette_right="p1000_single_v3.3",
)
pip_mounts = [OT3Mount.from_mount(m) for m, p in api.hardware_pipettes.items() if p]
while True:
for mount in pip_mounts:
pascals = await helpers_ot3.get_pressure_ot3(api, mount)
pico_farads = await helpers_ot3.get_capacitance_ot3(api, mount)
celsius, humidity = await helpers_ot3.get_temperature_humidity_ot3(
api, mount
)
print(
f"-----\n"
f"{mount.name}:\n"
f"\tpascals={pascals}\n"
f"\tpico_farads={pico_farads}\n"
f"\tcelsius={celsius}\n"
f"\thumidity={humidity}"
)
sleep(0.2)


if __name__ == "__main__":
parser = argparse.ArgumentParser()
parser.add_argument("--simulate", action="store_true")
args = parser.parse_args()
asyncio.run(_main(args.simulate))
18 changes: 0 additions & 18 deletions hardware-testing/hardware_testing/execute/README.md

This file was deleted.

4 changes: 0 additions & 4 deletions hardware-testing/hardware_testing/execute/__init__.py

This file was deleted.

Loading

0 comments on commit 055fafa

Please sign in to comment.