Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf(app,robot-server): Download analyses as raw JSON documents #13425

Merged
Show file tree
Hide file tree
Changes from 35 commits
Commits
Show all changes
37 commits
Select commit Hold shift + click to select a range
bc0abd4
Hack up some performance tests.
SyntaxColoring Aug 24, 2023
2bb3af4
temporarily remove tracking from run card, and check equipment from a…
b-cooper Aug 29, 2023
341eaf7
server initializing empty state for recent run protocol card
b-cooper Aug 29, 2023
5c4bcb7
remove old data fetching comments from required hardware hook
b-cooper Aug 29, 2023
6e3a261
Delete benchmarking script.
SyntaxColoring Aug 29, 2023
5cb7f3d
Merge branch 'chore_release-7.0.0' into performance_testing_analyses_…
SyntaxColoring Aug 30, 2023
0db671d
Add a migration to add the new column.
SyntaxColoring Aug 28, 2023
5965e92
Try to migrate records eagerly.
SyntaxColoring Aug 29, 2023
aeb5012
refactor useMostRecentCompletedAnalysis to use asDocument endpoint
b-cooper Aug 30, 2023
7c0f29d
back off the top level maintenance run poll a bit
b-cooper Aug 30, 2023
09eaf6e
Add a Tavern test for the analysis endpoints.
SyntaxColoring Aug 30, 2023
5d83c8b
Tweak serialization configurables to match what we normally do throug…
SyntaxColoring Aug 30, 2023
007d53e
Add endpoint docs and fix Content-Type header.
SyntaxColoring Aug 30, 2023
1211d1c
Unrelated test fixups.
SyntaxColoring Aug 30, 2023
9bc8b91
Add CompletedAnalysisStore unit tests.
SyntaxColoring Aug 30, 2023
0bcbb3e
Fix edge case with potentially NULL documents.
SyntaxColoring Aug 30, 2023
11c534f
update all instances of analyses query to as doc
b-cooper Aug 30, 2023
d2c4b04
undo problematic useprotocoldetailsforrun test fix
b-cooper Aug 30, 2023
f890c9f
fix up test for use protocol details for run
b-cooper Aug 30, 2023
b413c88
fix up formatting and tests
b-cooper Aug 30, 2023
00e9f0d
Import fixup.
SyntaxColoring Aug 31, 2023
1807fb7
Add checks for new endpoint in persistence test.
SyntaxColoring Aug 31, 2023
690d406
Add checks for new endpoint in persistence snapshot compatibility test.
SyntaxColoring Aug 31, 2023
954287e
Describe confusing schema stamp behavior.
SyntaxColoring Aug 31, 2023
67ecc5f
Simpler solution for the upgrade-downgrade-upgrade edge case.
SyntaxColoring Aug 31, 2023
8d62ad3
Document column data types.
SyntaxColoring Aug 31, 2023
2a4c36f
Format 'n lint.
SyntaxColoring Aug 31, 2023
afa7124
Update test_tables.py.
SyntaxColoring Aug 31, 2023
663f6d8
Delete test_migrations.py. :o
SyntaxColoring Aug 31, 2023
a639e8e
Note potential future trap with _systemd_notify().
SyntaxColoring Aug 31, 2023
9296728
Raise 404 if the given analysis is still pending.
SyntaxColoring Aug 31, 2023
ae61097
Minor fixups to comments and formatting.
SyntaxColoring Aug 31, 2023
593d509
we are all children of the machine. the machine takes care of us.
SyntaxColoring Aug 31, 2023
9ebe737
Bump copmatibility tests' startup timeouts, for CI.
SyntaxColoring Aug 31, 2023
ab8adac
Add missing router tests.
SyntaxColoring Aug 31, 2023
fe9adcc
Fix test docstrings.
SyntaxColoring Aug 31, 2023
6375ed1
It would help if I applied the increased timeout to the correct file.
SyntaxColoring Aug 31, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 18 additions & 0 deletions api-client/src/protocols/getProtocolAnalysisAsDocument.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
import { GET, request } from '../request'

import type { ResponsePromise } from '../request'
import type { HostConfig } from '../types'
import type { CompletedProtocolAnalysis } from '@opentrons/shared-data'

export function getProtocolAnalysisAsDocument(
config: HostConfig,
protocolId: string,
analysisId: string
): ResponsePromise<CompletedProtocolAnalysis> {
return request<CompletedProtocolAnalysis>(
GET,
`/protocols/${protocolId}/analyses/${analysisId}/asDocument`,
null,
config
)
}
1 change: 1 addition & 0 deletions api-client/src/protocols/index.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
export { getProtocol } from './getProtocol'
export { getProtocolAnalyses } from './getProtocolAnalyses'
export { getProtocolAnalysisAsDocument } from './getProtocolAnalysisAsDocument'
export { deleteProtocol } from './deleteProtocol'
export { createProtocol } from './createProtocol'
export { getProtocols } from './getProtocols'
Expand Down
1 change: 1 addition & 0 deletions app/src/assets/localization/en/device_details.json
Original file line number Diff line number Diff line change
Expand Up @@ -118,6 +118,7 @@
"run_again": "Run again",
"run_duration": "Run duration",
"serial_number": "Serial Number",
"robot_initializing": "Initializing...",
"set_block_temp": "Set temperature",
"set_block_temperature": "Set block temperature",
"set_engage_height": "Set Engage Height",
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,12 @@ import {
} from '@opentrons/components'

import { FloatingActionButton } from '..'
import { i18n } from '../../../i18n'

const render = (props: React.ComponentProps<typeof FloatingActionButton>) => {
return renderWithProviders(<FloatingActionButton {...props} />)[0]
return renderWithProviders(<FloatingActionButton {...props} />, {
i18nInstance: i18n,
})[0]
}

describe('FloatingActionButton', () => {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,15 +3,16 @@ import { UseQueryResult } from 'react-query'
import { renderHook } from '@testing-library/react-hooks'

import {
useProtocolAnalysesQuery,
useProtocolAnalysisAsDocumentQuery,
useProtocolQuery,
useRunQuery,
} from '@opentrons/react-api-client'

import { useProtocolAnalysisErrors } from '..'

import { RUN_ID_2 } from '../../../../organisms/RunTimeControl/__fixtures__'

import type { Run, ProtocolAnalyses } from '@opentrons/api-client'
import type { Run, Protocol } from '@opentrons/api-client'
import type {
CompletedProtocolAnalysis,
PendingProtocolAnalysis,
Expand All @@ -20,20 +21,27 @@ import type {
jest.mock('@opentrons/react-api-client')

const mockUseRunQuery = useRunQuery as jest.MockedFunction<typeof useRunQuery>
const mockUseProtocolAnalysesQuery = useProtocolAnalysesQuery as jest.MockedFunction<
typeof useProtocolAnalysesQuery

const mockUseProtocolQuery = useProtocolQuery as jest.MockedFunction<
typeof useProtocolQuery
>
const mockUseProtocolAnalysisAsDocumentQuery = useProtocolAnalysisAsDocumentQuery as jest.MockedFunction<
typeof useProtocolAnalysisAsDocumentQuery
>

describe('useProtocolAnalysisErrors hook', () => {
beforeEach(() => {
when(mockUseRunQuery)
.calledWith(null, { staleTime: Infinity })
.mockReturnValue({} as UseQueryResult<Run>)
when(mockUseProtocolAnalysesQuery)
.calledWith(null, { staleTime: Infinity })
when(mockUseProtocolQuery)
.calledWith(null)
.mockReturnValue({} as UseQueryResult<Protocol>)
when(mockUseProtocolAnalysisAsDocumentQuery)
.calledWith(null, null, { enabled: false })
.mockReturnValue({
data: { data: [] } as any,
} as UseQueryResult<ProtocolAnalyses>)
data: null,
} as UseQueryResult<CompletedProtocolAnalysis | null>)
})

afterEach(() => {
Expand Down Expand Up @@ -63,12 +71,18 @@ describe('useProtocolAnalysisErrors hook', () => {
.mockReturnValue({
data: { data: { protocolId: PROTOCOL_ID } } as any,
} as UseQueryResult<Run>)
when(mockUseProtocolAnalysesQuery)
.calledWith(PROTOCOL_ID, { staleTime: Infinity })
when(mockUseProtocolQuery)
.calledWith(PROTOCOL_ID)
.mockReturnValue({
data: { data: [PROTOCOL_ANALYSIS as any] },
} as UseQueryResult<ProtocolAnalyses>)

data: {
data: { analysisSummaries: [{ id: PROTOCOL_ANALYSIS.id }] },
} as any,
} as UseQueryResult<Protocol>)
when(mockUseProtocolAnalysisAsDocumentQuery)
.calledWith(PROTOCOL_ID, PROTOCOL_ANALYSIS.id, { enabled: true })
.mockReturnValue({
data: PROTOCOL_ANALYSIS,
} as UseQueryResult<CompletedProtocolAnalysis>)
const { result } = renderHook(() => useProtocolAnalysisErrors(RUN_ID_2))
expect(result.current).toStrictEqual({
analysisErrors: null,
Expand All @@ -86,12 +100,18 @@ describe('useProtocolAnalysisErrors hook', () => {
.mockReturnValue({
data: { data: { protocolId: PROTOCOL_ID } } as any,
} as UseQueryResult<Run>)
when(mockUseProtocolAnalysesQuery)
.calledWith(PROTOCOL_ID, { staleTime: Infinity })
when(mockUseProtocolQuery)
.calledWith(PROTOCOL_ID)
.mockReturnValue({
data: { data: [PROTOCOL_ANALYSIS as any] },
} as UseQueryResult<ProtocolAnalyses>)

data: {
data: { analysisSummaries: [{ id: PROTOCOL_ANALYSIS.id }] },
} as any,
} as UseQueryResult<Protocol>)
when(mockUseProtocolAnalysisAsDocumentQuery)
.calledWith(PROTOCOL_ID, PROTOCOL_ANALYSIS.id, { enabled: true })
.mockReturnValue({
data: PROTOCOL_ANALYSIS,
} as UseQueryResult<CompletedProtocolAnalysis>)
const { result } = renderHook(() => useProtocolAnalysisErrors(RUN_ID_2))
expect(result.current).toStrictEqual({
analysisErrors: null,
Expand All @@ -110,12 +130,22 @@ describe('useProtocolAnalysisErrors hook', () => {
.mockReturnValue({
data: { data: { protocolId: PROTOCOL_ID } } as any,
} as UseQueryResult<Run>)
when(mockUseProtocolAnalysesQuery)
.calledWith(PROTOCOL_ID, { staleTime: Infinity })
when(mockUseProtocolQuery)
.calledWith(PROTOCOL_ID)
.mockReturnValue({
data: { data: [PROTOCOL_ANALYSIS_WITH_ERRORS as any] },
} as UseQueryResult<ProtocolAnalyses>)

data: {
data: {
analysisSummaries: [{ id: PROTOCOL_ANALYSIS_WITH_ERRORS.id }],
},
} as any,
} as UseQueryResult<Protocol>)
when(mockUseProtocolAnalysisAsDocumentQuery)
.calledWith(PROTOCOL_ID, PROTOCOL_ANALYSIS_WITH_ERRORS.id, {
enabled: true,
})
.mockReturnValue({
data: PROTOCOL_ANALYSIS_WITH_ERRORS,
} as UseQueryResult<CompletedProtocolAnalysis>)
const { result } = renderHook(() => useProtocolAnalysisErrors(RUN_ID_2))
expect(result.current).toStrictEqual({
analysisErrors: [{ detail: 'fake error' }],
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ import { UseQueryResult } from 'react-query'
import { renderHook } from '@testing-library/react-hooks'

import {
useProtocolAnalysesQuery,
useProtocolAnalysisAsDocumentQuery,
useProtocolQuery,
useRunQuery,
} from '@opentrons/react-api-client'
Expand All @@ -12,25 +12,32 @@ import { useProtocolDetailsForRun } from '..'

import { RUN_ID_2 } from '../../../../organisms/RunTimeControl/__fixtures__'

import type { Protocol, Run, ProtocolAnalyses } from '@opentrons/api-client'
import type { Protocol, Run } from '@opentrons/api-client'
import { CompletedProtocolAnalysis } from '@opentrons/shared-data'

jest.mock('@opentrons/react-api-client')

const mockUseProtocolQuery = useProtocolQuery as jest.MockedFunction<
typeof useProtocolQuery
>
const mockUseProtocolAnalysesQuery = useProtocolAnalysesQuery as jest.MockedFunction<
typeof useProtocolAnalysesQuery
const mockUseProtocolAnalysisAsDocumentQuery = useProtocolAnalysisAsDocumentQuery as jest.MockedFunction<
typeof useProtocolAnalysisAsDocumentQuery
>
const mockUseRunQuery = useRunQuery as jest.MockedFunction<typeof useRunQuery>

const PROTOCOL_ID = 'fake_protocol_id'
const PROTOCOL_ANALYSIS = {
id: 'fake analysis',
status: 'completed',
labware: [],
} as any
const PROTOCOL_RESPONSE = {
data: {
protocolType: 'json',
createdAt: 'now',
id: '1',
id: PROTOCOL_ID,
metadata: { protocolName: 'fake protocol' },
analysisSummaries: [{ id: 'fake analysis', status: 'completed' }],
analysisSummaries: [{ id: PROTOCOL_ANALYSIS.id, status: 'completed' }],
key: 'fakeProtocolKey',
},
} as Protocol
Expand All @@ -43,11 +50,11 @@ describe('useProtocolDetailsForRun hook', () => {
when(mockUseProtocolQuery)
.calledWith(null, { staleTime: Infinity })
.mockReturnValue({} as UseQueryResult<Protocol>)
when(mockUseProtocolAnalysesQuery)
.calledWith(null, { staleTime: Infinity }, true)
when(mockUseProtocolAnalysisAsDocumentQuery)
.calledWith(null, null, { enabled: false, refetchInterval: 5000 })
.mockReturnValue({
data: { data: [] } as any,
} as UseQueryResult<ProtocolAnalyses>)
data: null,
} as UseQueryResult<CompletedProtocolAnalysis | null>)
})

afterEach(() => {
Expand All @@ -66,12 +73,6 @@ describe('useProtocolDetailsForRun hook', () => {
})

it('returns the protocol file when given a run id', async () => {
const PROTOCOL_ID = 'fake_protocol_id'
const PROTOCOL_ANALYSIS = {
id: 'fake analysis',
status: 'completed',
labware: [],
} as any
when(mockUseRunQuery)
.calledWith(RUN_ID_2, { staleTime: Infinity })
.mockReturnValue({
Expand All @@ -80,11 +81,22 @@ describe('useProtocolDetailsForRun hook', () => {
when(mockUseProtocolQuery)
.calledWith(PROTOCOL_ID, { staleTime: Infinity })
.mockReturnValue({ data: PROTOCOL_RESPONSE } as UseQueryResult<Protocol>)
when(mockUseProtocolAnalysesQuery)
.calledWith(PROTOCOL_ID, { staleTime: Infinity }, expect.any(Boolean))
when(mockUseProtocolAnalysisAsDocumentQuery)
.calledWith(PROTOCOL_ID, 'fake analysis', {
enabled: true,
refetchInterval: 5000,
})
.mockReturnValue({
data: { data: [PROTOCOL_ANALYSIS as any] },
} as UseQueryResult<ProtocolAnalyses>)
data: PROTOCOL_ANALYSIS,
} as UseQueryResult<CompletedProtocolAnalysis | null>)
when(mockUseProtocolAnalysisAsDocumentQuery)
.calledWith(PROTOCOL_ID, 'fake analysis', {
enabled: false,
refetchInterval: 5000,
})
.mockReturnValue({
data: PROTOCOL_ANALYSIS,
} as UseQueryResult<CompletedProtocolAnalysis | null>)

const { result } = renderHook(() => useProtocolDetailsForRun(RUN_ID_2))

Expand Down
16 changes: 10 additions & 6 deletions app/src/organisms/Devices/hooks/useProtocolAnalysisErrors.ts
Original file line number Diff line number Diff line change
@@ -1,6 +1,7 @@
import last from 'lodash/last'
import {
useProtocolAnalysesQuery,
useProtocolAnalysisAsDocumentQuery,
useProtocolQuery,
useRunQuery,
} from '@opentrons/react-api-client'

Expand All @@ -15,16 +16,19 @@ export function useProtocolAnalysisErrors(
): ProtocolAnalysisErrors {
const { data: runRecord } = useRunQuery(runId, { staleTime: Infinity })
const protocolId = runRecord?.data?.protocolId ?? null
const { data: protocolAnalyses } = useProtocolAnalysesQuery(protocolId, {
staleTime: Infinity,
})
const { data: protocolData } = useProtocolQuery(protocolId)
const {
data: mostRecentAnalysis,
} = useProtocolAnalysisAsDocumentQuery(
protocolId,
last(protocolData?.data.analysisSummaries)?.id ?? null,
{ enabled: protocolData != null }
)

if (protocolId === null || runRecord?.data?.current === false) {
return { analysisErrors: null }
}

const mostRecentAnalysis = last(protocolAnalyses?.data ?? []) ?? null

if (mostRecentAnalysis?.status !== 'completed') {
return { analysisErrors: null }
}
Expand Down
18 changes: 8 additions & 10 deletions app/src/organisms/Devices/hooks/useProtocolDetailsForRun.ts
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,8 @@ import last from 'lodash/last'
import { getRobotTypeFromLoadedLabware } from '@opentrons/shared-data'
import {
useProtocolQuery,
useProtocolAnalysesQuery,
useRunQuery,
useProtocolAnalysisAsDocumentQuery,
} from '@opentrons/react-api-client'

import type {
Expand All @@ -13,6 +13,7 @@ import type {
PendingProtocolAnalysis,
} from '@opentrons/shared-data'

const ANALYSIS_POLL_MS = 5000
export interface ProtocolDetails {
displayName: string | null
protocolData: CompletedProtocolAnalysis | PendingProtocolAnalysis | null
Expand All @@ -34,17 +35,15 @@ export function useProtocolDetailsForRun(
const { data: protocolRecord } = useProtocolQuery(protocolId, {
staleTime: Infinity,
})

const { data: protocolAnalyses } = useProtocolAnalysesQuery(
const { data: mostRecentAnalysis } = useProtocolAnalysisAsDocumentQuery(
protocolId,
last(protocolRecord?.data.analysisSummaries)?.id ?? null,
{
staleTime: Infinity,
},
isPollingProtocolAnalyses
enabled: protocolRecord != null && isPollingProtocolAnalyses,
refetchInterval: ANALYSIS_POLL_MS,
}
)

const mostRecentAnalysis = last(protocolAnalyses?.data ?? []) ?? null

React.useEffect(() => {
if (mostRecentAnalysis?.status === 'completed') {
setIsPollingProtocolAnalyses(false)
Expand All @@ -61,8 +60,7 @@ export function useProtocolDetailsForRun(
displayName: displayName ?? null,
protocolData: mostRecentAnalysis ?? null,
protocolKey: protocolRecord?.data.key ?? null,
isProtocolAnalyzing:
mostRecentAnalysis != null && mostRecentAnalysis?.status === 'pending',
isProtocolAnalyzing: protocolRecord != null && mostRecentAnalysis == null,
// this should be deleted as soon as analysis tells us intended robot type
robotType:
mostRecentAnalysis?.status === 'completed'
Expand Down
Original file line number Diff line number Diff line change
@@ -1,5 +1,7 @@
import last from 'lodash/last'
import {
useProtocolAnalysesQuery,
useProtocolAnalysisAsDocumentQuery,
useProtocolQuery,
useRunQuery,
} from '@opentrons/react-api-client'
import { CompletedProtocolAnalysis } from '@opentrons/shared-data'
Expand All @@ -9,14 +11,14 @@ export function useMostRecentCompletedAnalysis(
): CompletedProtocolAnalysis | null {
const { data: runRecord } = useRunQuery(runId)
const protocolId = runRecord?.data?.protocolId ?? null
const { data: protocolAnalyses } = useProtocolAnalysesQuery(protocolId)

return (
(protocolAnalyses?.data ?? [])
.reverse()
.find(
(analysis): analysis is CompletedProtocolAnalysis =>
analysis.status === 'completed'
) ?? null
const { data: protocolData } = useProtocolQuery(protocolId, {
enabled: protocolId != null,
})
const { data: analysis } = useProtocolAnalysisAsDocumentQuery(
protocolId,
last(protocolData?.data.analysisSummaries)?.id ?? null,
{ enabled: protocolData != null }
)

return analysis ?? null
}
Loading