Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(app,robot-server): Download analyses as raw JSON documents #13425

Merged
Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
bc0abd4
Hack up some performance tests.
SyntaxColoring Aug 24, 2023
2bb3af4
temporarily remove tracking from run card, and check equipment from a…
b-cooper Aug 29, 2023
341eaf7
server initializing empty state for recent run protocol card
b-cooper Aug 29, 2023
5c4bcb7
remove old data fetching comments from required hardware hook
b-cooper Aug 29, 2023
6e3a261
Delete benchmarking script.
SyntaxColoring Aug 29, 2023
5cb7f3d
Merge branch 'chore_release-7.0.0' into performance_testing_analyses_…
SyntaxColoring Aug 30, 2023
0db671d
Add a migration to add the new column.
SyntaxColoring Aug 28, 2023
5965e92
Try to migrate records eagerly.
SyntaxColoring Aug 29, 2023
aeb5012
refactor useMostRecentCompletedAnalysis to use asDocument endpoint
b-cooper Aug 30, 2023
7c0f29d
back off the top level maintenance run poll a bit
b-cooper Aug 30, 2023
09eaf6e
Add a Tavern test for the analysis endpoints.
SyntaxColoring Aug 30, 2023
5d83c8b
Tweak serialization configurables to match what we normally do throug…
SyntaxColoring Aug 30, 2023
007d53e
Add endpoint docs and fix Content-Type header.
SyntaxColoring Aug 30, 2023
1211d1c
Unrelated test fixups.
SyntaxColoring Aug 30, 2023
9bc8b91
Add CompletedAnalysisStore unit tests.
SyntaxColoring Aug 30, 2023
0bcbb3e
Fix edge case with potentially NULL documents.
SyntaxColoring Aug 30, 2023
11c534f
update all instances of analyses query to as doc
b-cooper Aug 30, 2023
d2c4b04
undo problematic useprotocoldetailsforrun test fix
b-cooper Aug 30, 2023
f890c9f
fix up test for use protocol details for run
b-cooper Aug 30, 2023
b413c88
fix up formatting and tests
b-cooper Aug 30, 2023
00e9f0d
Import fixup.
SyntaxColoring Aug 31, 2023
1807fb7
Add checks for new endpoint in persistence test.
SyntaxColoring Aug 31, 2023
690d406
Add checks for new endpoint in persistence snapshot compatibility test.
SyntaxColoring Aug 31, 2023
954287e
Describe confusing schema stamp behavior.
SyntaxColoring Aug 31, 2023
67ecc5f
Simpler solution for the upgrade-downgrade-upgrade edge case.
SyntaxColoring Aug 31, 2023
8d62ad3
Document column data types.
SyntaxColoring Aug 31, 2023
2a4c36f
Format 'n lint.
SyntaxColoring Aug 31, 2023
afa7124
Update test_tables.py.
SyntaxColoring Aug 31, 2023
663f6d8
Delete test_migrations.py. :o
SyntaxColoring Aug 31, 2023
a639e8e
Note potential future trap with _systemd_notify().
SyntaxColoring Aug 31, 2023
9296728
Raise 404 if the given analysis is still pending.
SyntaxColoring Aug 31, 2023
ae61097
Minor fixups to comments and formatting.
SyntaxColoring Aug 31, 2023
593d509
we are all children of the machine. the machine takes care of us.
SyntaxColoring Aug 31, 2023
9ebe737
Bump copmatibility tests' startup timeouts, for CI.
SyntaxColoring Aug 31, 2023
ab8adac
Add missing router tests.
SyntaxColoring Aug 31, 2023
fe9adcc
Fix test docstrings.
SyntaxColoring Aug 31, 2023
6375ed1
It would help if I applied the increased timeout to the correct file.
SyntaxColoring Aug 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions api-client/src/protocols/getProtocolAnalysisAsDocument.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { GET, request } from '../request'

import type { ResponsePromise } from '../request'
import type { HostConfig } from '../types'
import type { CompletedProtocolAnalysis } from '@opentrons/shared-data'

export function getProtocolAnalysisAsDocument(
config: HostConfig,
protocolId: string,
analysisId: string
): ResponsePromise<CompletedProtocolAnalysis> {
return request<CompletedProtocolAnalysis>(
GET,
`/protocols/${protocolId}/analyses/${analysisId}/asDocument`,
null,
config
)
}
1 change: 1 addition & 0 deletions api-client/src/protocols/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
export { getProtocol } from './getProtocol'
export { getProtocolAnalyses } from './getProtocolAnalyses'
export { getProtocolAnalysisAsDocument } from './getProtocolAnalysisAsDocument'
export { deleteProtocol } from './deleteProtocol'
export { createProtocol } from './createProtocol'
export { getProtocols } from './getProtocols'
Expand Down
1 change: 1 addition & 0 deletions app/src/assets/localization/en/device_details.json
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@
"run_again": "Run again",
"run_duration": "Run duration",
"serial_number": "Serial Number",
"robot_initializing": "Initializing...",
"set_block_temp": "Set temperature",
"set_block_temperature": "Set block temperature",
"set_engage_height": "Set Engage Height",
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import last from 'lodash/last'
import {
useProtocolAnalysesQuery,
useProtocolAnalysisAsDocumentQuery,
useProtocolQuery,
useRunQuery,
} from '@opentrons/react-api-client'
import { CompletedProtocolAnalysis } from '@opentrons/shared-data'
Expand All @@ -9,14 +11,14 @@ export function useMostRecentCompletedAnalysis(
): CompletedProtocolAnalysis | null {
const { data: runRecord } = useRunQuery(runId)
const protocolId = runRecord?.data?.protocolId ?? null
const { data: protocolAnalyses } = useProtocolAnalysesQuery(protocolId)

return (
(protocolAnalyses?.data ?? [])
.reverse()
.find(
(analysis): analysis is CompletedProtocolAnalysis =>
analysis.status === 'completed'
) ?? null
const { data: protocolData } = useProtocolQuery(protocolId, {
enabled: protocolId != null,
})
const { data: analysis } = useProtocolAnalysisAsDocumentQuery(
protocolId,
last(protocolData?.data.analysisSummaries)?.id ?? null,
{ enabled: protocolData != null }
)

return analysis ?? null
}
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,6 @@ import { useTrackEvent } from '../../../redux/analytics'
import { Skeleton } from '../../../atoms/Skeleton'
import { useMissingProtocolHardware } from '../../../pages/Protocols/hooks'
import { useCloneRun } from '../../ProtocolUpload/hooks'
import { useTrackProtocolRunEvent } from '../../Devices/hooks'
import { useMissingHardwareText } from './hooks'
import {
RUN_STATUS_FAILED,
Expand Down Expand Up @@ -74,7 +73,8 @@ export function ProtocolWithLastRun({
const isReadyToBeReRun = missingProtocolHardware.length === 0
const chipText = useMissingHardwareText(missingProtocolHardware)
const trackEvent = useTrackEvent()
const { trackProtocolRunEvent } = useTrackProtocolRunEvent(runData.id)
// TODO(BC, 08/29/23): reintroduce this analytics event when we refactor the hook to fetch data lazily (performance concern)
// const { trackProtocolRunEvent } = useTrackProtocolRunEvent(runData.id)
const onResetSuccess = (createRunResponse: Run): void =>
history.push(`runs/${createRunResponse.data.id}/setup`)
const { cloneRun } = useCloneRun(runData.id, onResetSuccess)
Expand Down Expand Up @@ -121,7 +121,8 @@ export function ProtocolWithLastRun({
name: 'proceedToRun',
properties: { sourceLocation: 'RecentRunProtocolCard' },
})
trackProtocolRunEvent({ name: 'runAgain' })
// TODO(BC, 08/29/23): reintroduce this analytics event when we refactor the hook to fetch data lazily (performance concern)
// trackProtocolRunEvent({ name: 'runAgain' })
}

const terminationTypeMap: { [runStatus in RunStatus]?: string } = {
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,40 @@
import * as React from 'react'
import { useTranslation } from 'react-i18next'

import {
Flex,
DIRECTION_COLUMN,
ALIGN_CENTER,
COLORS,
JUSTIFY_CENTER,
TYPOGRAPHY,
BORDERS,
Icon,
SPACING,
} from '@opentrons/components'

import { StyledText } from '../../../atoms/text'

export function ServerInitializing(): JSX.Element {
const { t } = useTranslation('device_details')
return (
<Flex
alignItems={ALIGN_CENTER}
backgroundColor={COLORS.darkBlack20}
flexDirection={DIRECTION_COLUMN}
height="27.25rem"
justifyContent={JUSTIFY_CENTER}
borderRadius={BORDERS.borderRadiusSize3}
gridGap={SPACING.spacing32}
>
<Icon name="ot-spinner" spin size="6rem" color={COLORS.darkBlack70} />
<StyledText
as="h4"
fontWeight={TYPOGRAPHY.fontWeightRegular}
color={COLORS.darkBlack70}
>
{t('robot_initializing')}
</StyledText>
</Flex>
)
}
3 changes: 2 additions & 1 deletion app/src/organisms/TakeoverModal/MaintenanceRunTakeover.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@ import {
import { TakeoverModal } from './TakeoverModal'
import { TakeoverModalContext } from './TakeoverModalContext'

const MAINTENANCE_RUN_POLL_MS = 10000
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Backing off this refetch interval to a slower poll. There are separate interval instantiated within wizards where we need more up to date info

interface MaintenanceRunTakeoverProps {
children: React.ReactNode
}
Expand All @@ -17,7 +18,7 @@ export function MaintenanceRunTakeover(
setIsODDMaintenanceInProgress,
] = React.useState<boolean>(false)
const maintenanceRunId = useCurrentMaintenanceRun({
refetchInterval: 5000,
refetchInterval: MAINTENANCE_RUN_POLL_MS,
}).data?.data.id
const isMaintenanceRunCurrent = maintenanceRunId != null

Expand Down
13 changes: 10 additions & 3 deletions app/src/pages/OnDeviceDisplay/ProtocolSetup/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,7 @@ import { useSelector } from 'react-redux'
import { useTranslation } from 'react-i18next'
import { useHistory, useParams } from 'react-router-dom'
import first from 'lodash/first'
import last from 'lodash/last'
import { css } from 'styled-components'

import { RUN_STATUS_IDLE, RUN_STATUS_STOPPED } from '@opentrons/api-client'
Expand All @@ -29,6 +30,7 @@ import {
useProtocolQuery,
useRunQuery,
useInstrumentsQuery,
useProtocolAnalysisAsDocumentQuery,
} from '@opentrons/react-api-client'
import {
getDeckDefFromRobotType,
Expand All @@ -46,7 +48,6 @@ import {
useAttachedModules,
useLPCDisabledReason,
} from '../../../organisms/Devices/hooks'
import { useMostRecentCompletedAnalysis } from '../../../organisms/LabwarePositionCheck/useMostRecentCompletedAnalysis'
import { getProtocolModulesInfo } from '../../../organisms/Devices/ProtocolRun/utils/getProtocolModulesInfo'
import { ProtocolSetupLabware } from '../../../organisms/ProtocolSetupLabware'
import { ProtocolSetupModules } from '../../../organisms/ProtocolSetupModules'
Expand Down Expand Up @@ -339,7 +340,13 @@ function PrepareToRun({
protocolRecord?.data.metadata.protocolName ??
protocolRecord?.data.files[0].name ??
''
const mostRecentAnalysis = useMostRecentCompletedAnalysis(runId)
const {
data: mostRecentAnalysis,
} = useProtocolAnalysisAsDocumentQuery(
protocolId,
last(protocolRecord?.data.analysisSummaries)?.id ?? null,
{ enabled: protocolRecord != null }
)
const { launchLPC, LPCWizard } = useLaunchLPC(runId)
const { setODDMaintenanceFlowInProgress } = useMaintenanceRunTakeover()

Expand Down Expand Up @@ -391,7 +398,7 @@ function PrepareToRun({
(protocolHasModules && attachedModules == null)

const speccedInstrumentCount =
mostRecentAnalysis !== null
mostRecentAnalysis != null
? mostRecentAnalysis.pipettes.length +
(getProtocolUsesGripper(mostRecentAnalysis) ? 1 : 0)
: 0
Expand Down
45 changes: 27 additions & 18 deletions app/src/pages/OnDeviceDisplay/RobotDashboard/index.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -21,12 +21,13 @@ import {
import { getOnDeviceDisplaySettings } from '../../../redux/config'
import { WelcomedModal } from './WelcomeModal'
import { RunData } from '@opentrons/api-client'
import { ServerInitializing } from '../../../organisms/OnDeviceDisplay/RobotDashboard/ServerInitializing'

export const MAXIMUM_RECENT_RUN_PROTOCOLS = 8

export function RobotDashboard(): JSX.Element {
const { t } = useTranslation('device_details')
const allRuns = useAllRunsQuery().data?.data ?? []
const { data: allRunsQueryData, error: allRunsQueryError } = useAllRunsQuery()

const { unfinishedUnboxingFlowRoute } = useSelector(
getOnDeviceDisplaySettings
Expand All @@ -35,7 +36,7 @@ export function RobotDashboard(): JSX.Element {
unfinishedUnboxingFlowRoute !== null
)

const recentRunsOfUniqueProtocols = allRuns
const recentRunsOfUniqueProtocols = (allRunsQueryData?.data ?? [])
.reverse() // newest runs first
.reduce<RunData[]>((acc, run) => {
if (
Expand All @@ -48,6 +49,29 @@ export function RobotDashboard(): JSX.Element {
}, [])
.slice(0, MAXIMUM_RECENT_RUN_PROTOCOLS)

let contents: JSX.Element = <EmptyRecentRun />
// GET runs query will error with 503 if database is initializing
// this should be momentary, and the type of error to come from this endpoint
// so, all errors will be mapped to an initializing spinner
if (allRunsQueryError != null) {
contents = <ServerInitializing />
} else if (recentRunsOfUniqueProtocols.length > 0) {
contents = (
<>
<StyledText
as="p"
fontWeight={TYPOGRAPHY.fontWeightSemiBold}
color={COLORS.darkBlack70}
>
{t('run_again')}
</StyledText>
<RecentRunProtocolCarousel
recentRunsOfUniqueProtocols={recentRunsOfUniqueProtocols}
/>
</>
)
}

return (
<Flex flexDirection={DIRECTION_COLUMN}>
<Navigation routes={onDeviceDisplayRoutes} />
Expand All @@ -59,22 +83,7 @@ export function RobotDashboard(): JSX.Element {
{showWelcomeModal ? (
<WelcomedModal setShowWelcomeModal={setShowWelcomeModal} />
) : null}
{recentRunsOfUniqueProtocols.length === 0 ? (
<EmptyRecentRun />
) : (
<>
<StyledText
as="p"
fontWeight={TYPOGRAPHY.fontWeightSemiBold}
color={COLORS.darkBlack70}
>
{t('run_again')}
</StyledText>
<RecentRunProtocolCarousel
recentRunsOfUniqueProtocols={recentRunsOfUniqueProtocols}
/>
</>
)}
{contents}
</Flex>
</Flex>
)
Expand Down
25 changes: 12 additions & 13 deletions app/src/pages/Protocols/hooks/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,6 +3,8 @@ import {
useInstrumentsQuery,
useModulesQuery,
useProtocolAnalysesQuery,
useProtocolAnalysisAsDocumentQuery,
useProtocolQuery,
} from '@opentrons/react-api-client'
import { getLabwareSetupItemGroups } from '../utils'

Expand Down Expand Up @@ -50,10 +52,12 @@ export type ProtocolHardware =
export const useRequiredProtocolHardware = (
protocolId: string
): { requiredProtocolHardware: ProtocolHardware[]; isLoading: boolean } => {
const { data: protocolAnalyses } = useProtocolAnalysesQuery(protocolId, {
staleTime: Infinity,
})
const mostRecentAnalysis = last(protocolAnalyses?.data ?? []) ?? null
const { data: protocolData } = useProtocolQuery(protocolId)
const { data: analysis } = useProtocolAnalysisAsDocumentQuery(
protocolId,
last(protocolData?.data.analysisSummaries)?.id ?? null,
{ enabled: protocolData != null }
)

const {
data: attachedModulesData,
Expand All @@ -67,16 +71,11 @@ export const useRequiredProtocolHardware = (
} = useInstrumentsQuery()
const attachedInstruments = attachedInstrumentsData?.data ?? []

if (
mostRecentAnalysis == null ||
mostRecentAnalysis?.status !== 'completed'
) {
if (analysis == null || analysis?.status !== 'completed') {
return { requiredProtocolHardware: [], isLoading: true }
}

const requiredGripper: ProtocolGripper[] = getProtocolUsesGripper(
mostRecentAnalysis
)
const requiredGripper: ProtocolGripper[] = getProtocolUsesGripper(analysis)
? [
{
hardwareType: 'gripper',
Expand All @@ -87,7 +86,7 @@ export const useRequiredProtocolHardware = (
]
: []

const requiredModules: ProtocolModule[] = mostRecentAnalysis.modules.map(
const requiredModules: ProtocolModule[] = analysis.modules.map(
({ location, model }) => {
return {
hardwareType: 'module',
Expand All @@ -99,7 +98,7 @@ export const useRequiredProtocolHardware = (
}
)

const requiredPipettes: ProtocolPipette[] = mostRecentAnalysis.pipettes.map(
const requiredPipettes: ProtocolPipette[] = analysis.pipettes.map(
({ mount, pipetteName }) => ({
hardwareType: 'pipette',
pipetteName: pipetteName,
Expand Down
1 change: 1 addition & 0 deletions react-api-client/src/protocols/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,6 @@ export { useAllProtocolsQuery } from './useAllProtocolsQuery'
export { useAllProtocolIdsQuery } from './useAllProtocolIdsQuery'
export { useProtocolQuery } from './useProtocolQuery'
export { useProtocolAnalysesQuery } from './useProtocolAnalysesQuery'
export { useProtocolAnalysisAsDocumentQuery } from './useProtocolAnalysisAsDocumentQuery'
export { useCreateProtocolMutation } from './useCreateProtocolMutation'
export { useDeleteProtocolMutation } from './useDeleteProtocolMutation'
Original file line number Diff line number Diff line change
@@ -0,0 +1,26 @@
import { UseQueryResult, useQuery } from 'react-query'
import { getProtocolAnalysisAsDocument } from '@opentrons/api-client'
import { useHost } from '../api'
import type { HostConfig } from '@opentrons/api-client'
import type { UseQueryOptions } from 'react-query'
import { CompletedProtocolAnalysis } from '@opentrons/shared-data'

export function useProtocolAnalysisAsDocumentQuery(
protocolId: string | null,
analysisId: string | null,
options?: UseQueryOptions<CompletedProtocolAnalysis>
): UseQueryResult<CompletedProtocolAnalysis | null> {
const host = useHost()
const query = useQuery<CompletedProtocolAnalysis>(
[host, 'protocols', protocolId, 'analyses', analysisId],
() =>
getProtocolAnalysisAsDocument(
host as HostConfig,
protocolId as string,
analysisId as string
).then(response => response.data),
options
)

return query
}
Loading