Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

refactor(api): Split UpdateCommandAction #14726

Merged
merged 24 commits into from
Mar 27, 2024
Merged

Conversation

SyntaxColoring
Copy link
Contributor

@SyntaxColoring SyntaxColoring commented Mar 25, 2024

Overview

Closes EXEC-340.

Changelog

  • Delete the monolithic UpdateCommandAction and replace it with new, distinct RunCommandAction and SucceedCommandAction command transitions. So we end up with this:

    stateDiagram-v2
        [*] --> queued: QueueCommandAction
        queued --> running: RunCommandAction
        running --> succeeded: SucceedCommandAction
        running --> failed: FailCommandAction
    
    Loading

    Compare with the old state diagram, where you could use UpdateCommandAction to jump around and skip over states.

  • Use asserts to enforce the preconditions of the state transitions. (For example, you can't run a command when a different one is already running.)

  • Make sure all completed commands have a notes list. If there are no notes, make it an empty list. Formerly, notes would not be present (None in Python, omitted in JSON) if there were no notes. Now, it's sort of like completedAt, where None means "hasn't happened yet."

    This entails a million billion test updates, unfortunately.

    This seemed like it would simplify things, but in hindsight, it doesn't make a big difference. I'm happy to undo this if anyone wants.

  • Update the LegacyContextPlugin. This was the really annoying part. Before, it emitted UpdateCommandActions to created succeeded and failed commands, skipping over queued and running. Now, because we're being stricter, it has to explicitly transition each command through queued and running.

    This bumps into a couple of ugly scary things, which I'll describe in separate threads below.

Test plan

  • Make sure legacy OT-2 protocols (PAPIv≤2.13, JSONv≤5) still work.
    • Make sure pausing and playing still works.
    • Make sure the run progress monitor in the Opentrons App still tracks the current step and shows the predicted future steps.
    • Make sure the robot still homes before the protocol begins.
  • Make sure Labware Position Check (a kind of live HTTP run) still works.
  • Make sure more modern protocols (PAPIv≥2.14, JSONv≥6) still work.

Review requests

See my review comments.

This diff got huge, sorry. Most of the files only had trivial changes.

Risk analysis

Medium. This touches some sensitive and messy code having to do with legacy (PAPIv≤2.13, JSONv≤5) protocols. Other than that, the changes are simple.

These are the easy ones, since they only care about command completions.
legacy
The exact sequence is unclear to me, but it's something like the PAPI thread trying to synthesize a running command while the engine is actually running the initial home. I don't know why this apparent concurrent robot control hasn't caused physical problems with robot motion.
@SyntaxColoring SyntaxColoring force-pushed the split_updatecommandaction branch from c2ccc93 to b5c54af Compare March 25, 2024 16:39
@sfoster1
Copy link
Member

After looking at it I think these are good concurrency hazards to fix and I think we can definitely do it especially if we're open to alternative means of fixing things. The home command looks pretty solved, and I think with the alternate intent we can handle the legacy python pretty well.

Simplify note update logic so it only overwrites, never appends. Now that UpdateCommandAction has been replaced by terminal SucceedCommandAction/FailCommandAction, appending can't happen in practice.

Refactor tests to test command notes as part of general success/failure, to save a bunch of test boilerplate.
Copy link

codecov bot commented Mar 26, 2024

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 67.18%. Comparing base (2857419) to head (33e39e0).
Report is 20 commits behind head on edge.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             edge   #14726      +/-   ##
==========================================
- Coverage   67.19%   67.18%   -0.02%     
==========================================
  Files        2495     2495              
  Lines       71563    71538      -25     
  Branches     9076     9076              
==========================================
- Hits        48088    48063      -25     
  Misses      21330    21330              
  Partials     2145     2145              
Flag Coverage Δ
g-code-testing 92.43% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files Coverage Δ
.../src/opentrons/protocol_engine/actions/__init__.py 100.00% <ø> (ø)
...i/src/opentrons/protocol_engine/actions/actions.py 100.00% <ø> (ø)
...rons/protocol_engine/execution/command_executor.py 100.00% <ø> (ø)
...i/src/opentrons/protocol_engine/protocol_engine.py 100.00% <ø> (ø)
...pi/src/opentrons/protocol_engine/state/commands.py 99.33% <ø> (-0.03%) ⬇️
api/src/opentrons/protocol_engine/state/labware.py 100.00% <ø> (ø)
api/src/opentrons/protocol_engine/state/modules.py 91.25% <ø> (-0.05%) ⬇️
...pi/src/opentrons/protocol_engine/state/pipettes.py 100.00% <ø> (ø)
...opentrons/protocol_runner/legacy_command_mapper.py 98.23% <ø> (-0.11%) ⬇️
...opentrons/protocol_runner/legacy_context_plugin.py 100.00% <ø> (ø)
... and 1 more

@SyntaxColoring SyntaxColoring force-pushed the split_updatecommandaction branch from ca56e85 to e284a7a Compare March 26, 2024 23:20
@SyntaxColoring SyntaxColoring force-pushed the split_updatecommandaction branch from e284a7a to 8c6f65d Compare March 27, 2024 06:34
@SyntaxColoring SyntaxColoring marked this pull request as ready for review March 27, 2024 07:01
@SyntaxColoring SyntaxColoring requested review from a team as code owners March 27, 2024 07:01
Copy link
Member

@sfoster1 sfoster1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good in structure with caveats inline.

I'm a little curious about the changes in the diff to add notes: [] everywhere. I don't think you made that value non optional in the command dataclass here - why add all that stuff but keep it optional?

"""Mark a given command as succeeded.

At the time of dispatching this action, the command must be running.
"""

command: Command
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

when UpdateCommandAction was the everything-doer, having the command embedded made sense, but now having it in SucceedCommandAction looks a little weirder amongst all its siblings that only reference the id. Does it maybe make sense to have SucceedCommandAction carry notes: List[CommandNote], result: CommandResult, private_result: CommandPrivateResult independently?

Copy link
Contributor

@TamarZanzouri TamarZanzouri Mar 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree with @sfoster1 we might want to add these to the SucceedCommandAction. I agree UpdateCommandAction had too many responsibilities

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yep, that makes sense.

I’m pretty sure some of the states stores are reading command.params, too. So to replace that, we’d either need to add params to CompleteCommandAction, or give every state store access to the CommandView so they can retrieve the params themselves.

"""Mark a given command as failed.

At the time of dispatching this action, the command must be running.
"""

command_id: str
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we don't want to make SucceedCommandAction carry everything independently, maybe we should make FailCommandAction embed the command too. Both of these actions are going to cause pretty equivalent modifications to the command dataclass anyway.


self._action_dispatcher.dispatch(
FailCommandAction(
error=error,
command_id=command_id,
error_id=self._model_utils.generate_id(),
failed_at=self._model_utils.get_timestamp(),
notes=note_tracker.get_notes(),
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

these notes changes means that command notes from before the beginning of execution will be lost. this might be fine, but we should note (hehe) it somewhere

@SyntaxColoring
Copy link
Contributor Author

I'm a little curious about the changes in the diff to add notes: [] everywhere. I don't think you made that value non optional in the command dataclass here - why add all that stuff but keep it optional?

The idea was to keep it None/omitted until the command completes, at which point it becomes the real final list.

I don’t know if this was a good idea.

Copy link
Contributor

@TamarZanzouri TamarZanzouri left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changes make sense! I prefer dispatching specific actions instead of one action to accommodate all these changes. lets talk more about hoisting the commands queue in the task_queue although looks like you brought us closer there too :-)

@SyntaxColoring SyntaxColoring merged commit d0d17d7 into edge Mar 27, 2024
22 checks passed
@SyntaxColoring SyntaxColoring deleted the split_updatecommandaction branch March 27, 2024 20:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants