Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

fix(api): filter get_run_commands #16001

31 changes: 29 additions & 2 deletions api/src/opentrons/protocol_engine/state/command_history.py
Original file line number Diff line number Diff line change
Expand Up @@ -101,9 +101,14 @@ def get_all_ids(self) -> List[str]:
"""Get all command IDs."""
return self._all_command_ids

def get_slice(self, start: int, stop: int) -> List[Command]:
def get_slice(
self, start: int, stop: int, command_ids: Optional[list[str]] = None
) -> List[Command]:
"""Get a list of commands between start and stop."""
commands = self._all_command_ids[start:stop]
selected_command_ids = (
command_ids if command_ids is not None else self._all_command_ids
)
commands = selected_command_ids[start:stop]
return [self._commands_by_id[command].command for command in commands]

def get_tail_command(self) -> Optional[CommandEntry]:
Expand All @@ -127,6 +132,28 @@ def get_running_command(self) -> Optional[CommandEntry]:
else:
return self._commands_by_id[self._running_command_id]

def get_filtered_command_ids(
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we add protocol intent filter? probably?

self,
command_intents: list[CommandIntent] = [
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should we change this to bool and add the logic within?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it depends on how the performance questions (in my other comment) get solved. You may find that it's too much work to get it to accept arbitrary CommandIntents, and we can only filter FIXIT for now.

So I guess let's focus on that question and then come back to this.

CommandIntent.PROTOCOL,
CommandIntent.SETUP,
],
) -> List[str]:
filtered_command = self._commands_by_id
if CommandIntent.FIXIT not in command_intents:
filtered_command = {
key: val
for key, val in self._commands_by_id.items()
if val.command.intent != CommandIntent.FIXIT
}
if CommandIntent.SETUP not in command_intents:
filtered_command = {
key: val
for key, val in filtered_command.items()
if val.command.intent != CommandIntent.SETUP
}
return list(filtered_command.keys())
Comment on lines +142 to +155
Copy link
Contributor

@SyntaxColoring SyntaxColoring Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm concerned about the performance of this implementation.

Formerly, if an HTTP client asked to retrieve n commands, we could do it in average O(n) time. Now, if a client asks to retrieve n filtered commands, it will take O(m) time, where m is the number of total commands in the run. This could be quite bad if the run is long and the client is polling us for the run log.

I think before merging, we need to confirm with testing that response times will stay acceptable even with, say, a 10,000-command protocol. (I'm getting the number 10,000 from here—it's pessimistic but realistic.)

A more performant way to implement this will probably involve some kind of speedup index, like we do for _queued_setup_command_ids, etc. It may also involve combining the filtering and slicing steps.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I agree. I told Jamey that I am concerned from the performance as well. The problem is that we have a fixit_queued_commad_ids but once it's executed it's not in the queue anymore, it's just in the commands queue. This was my first implementation but found bugs related to that. I will explore other options.


def get_queue_ids(self) -> OrderedSet[str]:
"""Get the IDs of all queued protocol commands, in FIFO order."""
return self._queued_command_ids
Expand Down
20 changes: 15 additions & 5 deletions api/src/opentrons/protocol_engine/state/commands.py
Original file line number Diff line number Diff line change
Expand Up @@ -580,18 +580,25 @@ def get_all(self) -> List[Command]:
return self._state.command_history.get_all_commands()

def get_slice(
self,
cursor: Optional[int],
length: int,
self, cursor: Optional[int], length: int, include_fixit_commands: bool
) -> CommandSlice:
"""Get a subset of commands around a given cursor.

If the cursor is omitted, a cursor will be selected automatically
based on the currently running or most recently executed command.
"""
command_ids = self._state.command_history.get_filtered_command_ids(
command_intents=[
CommandIntent.PROTOCOL,
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

kind of hate this. should we just change this to a list of intents in robot server?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm totally neutral about it, but I think this also depends on the performance questions in CommandHistory (see my other comment). So let's come back to this.

CommandIntent.SETUP,
CommandIntent.FIXIT,
]
if include_fixit_commands
else [CommandIntent.PROTOCOL, CommandIntent.SETUP]
)
running_command = self._state.command_history.get_running_command()
queued_command_ids = self._state.command_history.get_queue_ids()
total_length = self._state.command_history.length()
total_length = len(command_ids)

# TODO(mm, 2024-05-17): This looks like it's attempting to do the same thing
# as self.get_current(), but in a different way. Can we unify them?
Expand Down Expand Up @@ -620,7 +627,10 @@ def get_slice(
# start is inclusive, stop is exclusive
actual_cursor = max(0, min(cursor, total_length - 1))
stop = min(total_length, actual_cursor + length)
commands = self._state.command_history.get_slice(start=actual_cursor, stop=stop)

commands = self._state.command_history.get_slice(
start=actual_cursor, stop=stop, command_ids=command_ids
)

return CommandSlice(
commands=commands,
Expand Down
7 changes: 3 additions & 4 deletions api/src/opentrons/protocol_runner/run_orchestrator.py
Original file line number Diff line number Diff line change
Expand Up @@ -257,18 +257,17 @@ def get_current_command(self) -> Optional[CommandPointer]:
return self._protocol_engine.state_view.commands.get_current()

def get_command_slice(
self,
cursor: Optional[int],
length: int,
self, cursor: Optional[int], length: int, include_fixit_commands: bool
) -> CommandSlice:
"""Get a slice of run commands.

Args:
cursor: Requested index of first command in the returned slice.
length: Length of slice to return.
all_commands: Get all command intents.
"""
return self._protocol_engine.state_view.commands.get_slice(
cursor=cursor, length=length
cursor=cursor, length=length, include_fixit_commands=include_fixit_commands
)

def get_command_error_slice(
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,14 @@


def create_queued_command_entry(
command_id: str = "command-id", index: int = 0
command_id: str = "command-id",
intent: CommandIntent = CommandIntent.PROTOCOL,
index: int = 0,
) -> CommandEntry:
"""Create a command entry for a queued command."""
return CommandEntry(create_queued_command(command_id=command_id), index)
return CommandEntry(
create_queued_command(command_id=command_id, intent=intent), index
)


def create_fixit_command_entry(
Expand Down Expand Up @@ -94,6 +98,51 @@ def test_get_all_commands(command_history: CommandHistory) -> None:
]


def test_get_filtered_commands(command_history: CommandHistory) -> None:
"""It should return a list of all commands without fixit commands."""
assert list(command_history.get_filtered_command_ids()) == []
command_entry_1 = create_queued_command(command_id="0")
command_entry_2 = create_queued_command(command_id="1")
fixit_command_entry_1 = create_queued_command(
intent=CommandIntent.FIXIT, command_id="fixit-1"
)
command_history.append_queued_command(command_entry_1)
command_history.append_queued_command(command_entry_2)
command_history.append_queued_command(fixit_command_entry_1)
assert list(command_history.get_filtered_command_ids()) == ["0", "1"]


def test_get_all_filtered_commands(command_history: CommandHistory) -> None:
"""It should return a list of all commands without fixit commands."""
assert (
list(
command_history.get_filtered_command_ids(
command_intents=[
CommandIntent.FIXIT,
CommandIntent.PROTOCOL,
CommandIntent.SETUP,
]
)
)
== []
)
command_entry_1 = create_queued_command_entry()
command_entry_2 = create_queued_command_entry(index=1, intent=CommandIntent.SETUP)
fixit_command_entry_1 = create_queued_command_entry(intent=CommandIntent.FIXIT)
command_history._add("0", command_entry_1)
command_history._add("1", command_entry_2)
command_history._add("fixit-1", fixit_command_entry_1)
assert list(
command_history.get_filtered_command_ids(
command_intents=[
CommandIntent.FIXIT,
CommandIntent.PROTOCOL,
CommandIntent.SETUP,
]
)
) == ["0", "1", "fixit-1"]


def test_get_all_ids(command_history: CommandHistory) -> None:
"""It should return a list of all command IDs."""
assert command_history.get_all_ids() == []
Expand All @@ -119,6 +168,21 @@ def test_get_slice(command_history: CommandHistory) -> None:
]


def test_get_slice_with_filtered_list(command_history: CommandHistory) -> None:
"""It should return a slice of filtered commands."""
assert command_history.get_slice(0, 2) == []
command_entry_1 = create_queued_command_entry()
command_entry_2 = create_queued_command_entry(index=1)
command_entry_3 = create_queued_command_entry(index=2)
command_history._add("0", command_entry_1)
command_history._add("1", command_entry_2)
command_history._add("2", command_entry_3)
filtered_list = ["0", "1"]
assert command_history.get_slice(1, 3, command_ids=filtered_list) == [
command_entry_2.command,
]


def test_get_tail_command(command_history: CommandHistory) -> None:
"""It should return the tail command."""
assert command_history.get_tail_command() is None
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -904,7 +904,7 @@ def test_get_current() -> None:
def test_get_slice_empty() -> None:
"""It should return a slice from the tail if no current command."""
subject = get_command_view(commands=[])
result = subject.get_slice(cursor=0, length=2)
result = subject.get_slice(cursor=0, length=2, include_fixit_commands=True)

assert result == CommandSlice(commands=[], cursor=0, total_length=0)

Expand All @@ -918,15 +918,15 @@ def test_get_slice() -> None:

subject = get_command_view(commands=[command_1, command_2, command_3, command_4])

result = subject.get_slice(cursor=1, length=3)
result = subject.get_slice(cursor=1, length=3, include_fixit_commands=True)

assert result == CommandSlice(
commands=[command_2, command_3, command_4],
cursor=1,
total_length=4,
)

result = subject.get_slice(cursor=-3, length=10)
result = subject.get_slice(cursor=-3, length=10, include_fixit_commands=True)

assert result == CommandSlice(
commands=[command_1, command_2, command_3, command_4],
Expand All @@ -944,7 +944,7 @@ def test_get_slice_default_cursor_no_current() -> None:

subject = get_command_view(commands=[command_1, command_2, command_3, command_4])

result = subject.get_slice(cursor=None, length=3)
result = subject.get_slice(cursor=None, length=3, include_fixit_commands=True)

assert result == CommandSlice(
commands=[command_2, command_3, command_4],
Expand Down Expand Up @@ -975,7 +975,7 @@ def test_get_slice_default_cursor_failed_command() -> None:
failed_command=CommandEntry(index=2, command=command_3),
)

result = subject.get_slice(cursor=None, length=3)
result = subject.get_slice(cursor=None, length=3, include_fixit_commands=True)

assert result == CommandSlice(
commands=[command_3, command_4],
Expand All @@ -997,7 +997,7 @@ def test_get_slice_default_cursor_running() -> None:
running_command_id="command-id-3",
)

result = subject.get_slice(cursor=None, length=2)
result = subject.get_slice(cursor=None, length=2, include_fixit_commands=True)

assert result == CommandSlice(
commands=[command_3, command_4],
Expand All @@ -1006,6 +1006,51 @@ def test_get_slice_default_cursor_running() -> None:
)


def test_get_slice_without_fixit() -> None:
Copy link
Contributor

@SyntaxColoring SyntaxColoring Aug 14, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can we add this in test_command_state instead, if you have time? For the reasons described at the top of this file.

This is a lower-priority request because the CommandView.get_slice() tests have a lot of confusing overlap with the ones for CommandHistory and I would not mind if we just ignored it for now. :^)

"""It should select a cursor based on the running command, if present."""
command_1 = create_succeeded_command(command_id="command-id-1")
command_2 = create_succeeded_command(command_id="command-id-2")
command_3 = create_running_command(command_id="command-id-3")
command_4 = create_queued_command(command_id="command-id-4")
command_5 = create_queued_command(command_id="command-id-5")
command_6 = create_queued_command(
command_id="fixit-id-1", intent=cmd.CommandIntent.FIXIT
)
command_7 = create_queued_command(
command_id="fixit-id-2", intent=cmd.CommandIntent.FIXIT
)

subject = get_command_view(
commands=[
command_1,
command_2,
command_3,
command_4,
command_5,
command_6,
command_7,
],
queued_command_ids=[
"command-id-1",
"command-id-2",
"command-id-3",
"command-id-4",
"command-id-5",
"fixit-id-1",
"fixit-id-2",
],
queued_fixit_command_ids=["fixit-id-1", "fixit-id-2"],
)

result = subject.get_slice(cursor=None, length=7, include_fixit_commands=False)

assert result == CommandSlice(
commands=[command_1, command_2, command_3, command_4, command_5],
cursor=0,
total_length=5,
)


def test_get_errors_slice_empty() -> None:
"""It should return a slice from the tail if no current command."""
subject = get_command_view(failed_command_errors=[])
Expand Down
4 changes: 3 additions & 1 deletion robot-server/robot_server/commands/router.py
Original file line number Diff line number Diff line change
Expand Up @@ -154,7 +154,9 @@ async def get_commands_list(
cursor: Cursor index for the collection response.
pageLength: Maximum number of items to return.
"""
cmd_slice = orchestrator.get_command_slice(cursor=cursor, length=pageLength)
cmd_slice = orchestrator.get_command_slice(
cursor=cursor, length=pageLength, include_fixit_commands=True
)
commands = cast(List[StatelessCommand], cmd_slice.commands)
meta = MultiBodyMeta(cursor=cmd_slice.cursor, totalLength=cmd_slice.total_length)

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -226,7 +226,11 @@ def get_command_slice(
cursor: Requested index of first command in the returned slice.
length: Length of slice to return.
"""
return self.run_orchestrator.get_command_slice(cursor=cursor, length=length)
return self.run_orchestrator.get_command_slice(
cursor=cursor,
length=length,
include_fixit_commands=False,
)

def get_current_command(self) -> Optional[CommandPointer]:
"""Get the "current" command, if any."""
Expand Down
8 changes: 8 additions & 0 deletions robot-server/robot_server/runs/router/commands_router.py
Original file line number Diff line number Diff line change
Expand Up @@ -271,6 +271,11 @@ async def get_run_commands(
_DEFAULT_COMMAND_LIST_LENGTH,
description="The maximum number of commands in the list to return.",
),
includeFixitCommands: bool = Query(
True,
description="If `true`, return all commands (protocol, setup, fixit)."
" If `false`, only return safe commands (protocol, setup).",
),
run_data_manager: RunDataManager = Depends(get_run_data_manager),
) -> PydanticResponse[MultiBody[RunCommandSummary, CommandCollectionLinks]]:
"""Get a summary of a set of commands in a run.
Expand All @@ -279,13 +284,16 @@ async def get_run_commands(
runId: Requested run ID, from the URL
cursor: Cursor index for the collection response.
pageLength: Maximum number of items to return.
includeFixitCommands: If `true`, return all commands."
" If `false`, only return safe commands.
run_data_manager: Run data retrieval interface.
"""
try:
command_slice = run_data_manager.get_commands_slice(
run_id=runId,
cursor=cursor,
length=pageLength,
include_fixit_commands=includeFixitCommands,
)
except RunNotFoundError as e:
raise RunNotFound.from_exc(e).as_error(status.HTTP_404_NOT_FOUND) from e
Expand Down
6 changes: 5 additions & 1 deletion robot-server/robot_server/runs/run_data_manager.py
Original file line number Diff line number Diff line change
Expand Up @@ -371,20 +371,24 @@ def get_commands_slice(
run_id: str,
cursor: Optional[int],
length: int,
include_fixit_commands: bool,
) -> CommandSlice:
"""Get a slice of run commands.

Args:
run_id: ID of the run.
cursor: Requested index of first command in the returned slice.
length: Length of slice to return.
include_fixit_commands: Weather we should include fixit commands.

Raises:
RunNotFoundError: The given run identifier was not found in the database.
"""
if run_id == self._run_orchestrator_store.current_run_id:
return self._run_orchestrator_store.get_command_slice(
cursor=cursor, length=length
cursor=cursor,
length=length,
include_fixit_commands=include_fixit_commands,
)

# Let exception propagate
Expand Down
Loading
Loading