Volume Testing on the FHIR Core client app

[ndegwamartin]

Overview
We need to conduct Volume Testing to see how the app scales with a lot of data. This testing will help highlight the various bottlenecks with the app.

Initial Test Data

• 300 Households
• 10 Patients per house hold
• 30 Tasks per patient

We can gradually increase the data set size as we optimize and improve.

Tasks

• Set up backend infrastructure
• Set up user assignment data + Gateway configuration
• Set up the dummy test data
• Conduct volume testing and document results

[Mstjamush]

Device Specs

Phone - Techno Spark 7
RAM - 3GB
ROM - 64GB
Android Version - 11

WiFi - 20Mbps
Number of Households - 1025
Number of Clients - 10250
Number of tasks - 3075

Results

• Initial Sync - 00:39:19.84
• Loading "Add Household" form -00:00:02.57
• Saving a Household - 00:00:45.06
• Loading "Add HH member" form - 00:00:02.38
• Saving a HH Member - 00:00:09.51
• Load HH profile - 00:00.11.68
• Load the All Households Register - 00:00:21.83

[ellykits]

@Mstjamush Can you explicitly define the time format for the starts shared above? e.g. HH:MM:ss

[Mstjamush]

@ellykits it's in HH:MM:ss format

[pld]

@Mstjamush can you share the same results for 10k households please?

[Mstjamush]

@pld Let me generate the HHs and share

[ndegwamartin]

Findings investigating the MWCore app:

• Removing Order By filter e.g. Order by Patient.name improves the register performance significantly from 25 seconds to 35 milliseconds
• Getting the total resources' count used in paginating/progressing during initial sync has a significant latency

[ndegwamartin]

Observation on the Search Query:

1. It uses the same Index to sort by as the register
2. Removing the index improves the performance as well
3. Search performance latency is directly proportional to the size of the resulting data set

Opportunities for improvement

1. Change search index to a non String type index e.g. last updated
2. Remove the index entirely - search narrows down the records enough
3. Limit triggering the db search until a minimum limit of characters is input by the user e.g. 3 chars

[ndegwamartin]

Ticket to track performance optimizations on the FHIR SDK engine - google/android-fhir#2547

[ndegwamartin]

For the form loading, after profiling MWCore made the following observations for the Registration form:

1. First time load of the Registration form is quite fast. Subsequent attempts take 12 to 13 seconds
2. Loading the Questionnaire resource from the DB seems to introduce this latency

We can implement a caching mechanism for the resources once loaded the first time and figure out an appropriate cache invalidation mechanism. This can be informed by various factors like we don't expect Questionnaire resources to change frequently during use.

[LZRS]

For the page registers, after doing some further tests, we've realized that there are three types of queries that are run

1. Query to populate the actual registers, including subqueries used to populate specific fields
2. Query to get total count of pages, used for navigation at the bottom
3. Query to populate counts for the sidenav bar menu items

The last two are the most costly, since they'd need to go through entire db rows to get the total count.

Also, to note, is that all queries in the sdk are wrapped within db transactions. From Room's documentation

Room will only perform at most one transaction at a time, additional transactions are queued and executed on a first come, first serve order.

So, a query can seem to take long because it's queued and waiting for other transactions to finish. That is, regardless of whether the queries are in different coroutines, and were running concurrently.
For example, in our case assuming the 3 different query types are ran in 3 different coroutines, the order in which the transactions are called can be somewhat arbitrary and when in order of either 2 - 3 - 1 or 3 - 2 - 1 the query to populate the register would be the last one called and the register would seem to take too long to load/show

[LZRS]

Separate ticket to track select query being queued, instead of running concurrently: google/android-fhir#2552

[Mstjamush]

Device Specs

Phone - SAMSUNG A12
RAM - 3GB
ROM - 64GB
Android Version - 12

_WiFi - 20Mbps
Number of Clients - 7661

Results

• Loading the Find person Register - 1:36:33.93
• Search Find Person Register - 1:13:25.87
• Loading Register person form - 0:00:06.7
• Saving a Register person - 0:00:07.66
• Loading Record ART Number form - 0:00:03.36
• Saving a Record ART Number - 0:00:04.39
• Loading Clinical Stage and CD4 form - 0:02:21.29
• Saving a Clinical Stage and CD4 - 0:00:15.31
• Loading Welcome Service Newly Diagnosed form - 0:02:21.26
• Saving a Welcome Service Newly Diagnosed - 0:00:06.49
• Loading TB/Covid Screening - 0:00:07.61
• Saving TB/COVID screening - 0:00:02.61
• Loading Vitals Page -0:02:22.26
• Saving Vitals - 0:00:06.88
• Loading Index Case Testing - 0:01:30.72
• Saving Index Case Testing - 0:00:04.67
• Loading Guardian Regidtration - 0:02:24.05
• Saving Guardian Registration - 0:00:03.37
• Loading TB History and Regimen - 0:02:24.61
• Saving TB History and Regimen - 0:00:13.27

[Mstjamush]

Tests after refactors

Device Specs

Phone - SAMSUNG A12
RAM - 3GB
ROM - 64GB
Android Version - 12

_WiFi - 20Mbps
Number of Clients - 7661

Results

• Loading the Find person Register - 0:11:42.49
• Search Find Person Register - 0:16:47.76
• Loading Register person form - 0:00:02.87
• Saving a Register person - 0:00:06.14
• Loading Record ART Number form - 0:00:02.66
• Saving a Record ART Number - 0:00:04.39
• Loading Clinical Stage and CD4 form - 0:00:02.91
• Saving a Clinical Stage and CD4 - 0:00:3.82
• Loading Welcome Service Newly Diagnosed form - 0:00:03.16
• Saving a Welcome Service Newly Diagnosed - 0:00:05.16
• Loading TB/Covid Screening - 0:00:02.24
• Saving TB/COVID screening - 0:00:02.32
• Loading Vitals Page -0:00:10.16
• Saving Vitals - 0:00:03.62
• Loading Index Case Testing - 0:00:02.37
• Saving Index Case Testing - 0:00:04.52
• Loading Guardian Registration - 0:00:03.05
• Saving Guardian Registration - 0:00:02.90
• Loading TB History and Regimen - 0:00:04.27
• Saving TB History and Regimen - 0:00:02.96

[Mstjamush]

Tests after refactors

Device Specs

Phone - SAMSUNG A12
RAM - 3GB
ROM - 64GB
Android Version - 12

_WiFi - 20Mbps
Number of Clients - 7661

Results

• Loading the Find person Register - 0:00:27.05
• Search Find Person Register - 0:00:03.65
• One character search on Find Person Register - 0:00:02.97
• Two character search on Find Person Register - 0:00:02.62
• Loading Register person form - 0:00:05.71
• Saving a Register person - 0:00:12.73
• Loading Record ART Number form - 0:00:02.55
• Saving a Record ART Number - 0:00:03.24
• Loading Clinical Stage and CD4 form - 0:00:01.99
• Saving a Clinical Stage and CD4 - 0:00:2.68
• Loading Welcome Service Newly Diagnosed form - 0:00:02.66
• Saving a Welcome Service Newly Diagnosed - 0:00:04.88
• Loading TB/Covid Screening - 0:00:01.90
• Saving TB/COVID screening - 0:00:02.18
• Loading Vitals Page -0:00:10.98
• Saving Vitals - 0:00:04.78
• Loading Index Case Testing - 0:00:03.41
• Saving Index Case Testing - 0:00:04.62
• Loading Guardian Registration - 0:00:02.88
• Saving Guardian Registration - 0:00:03.08
• Loading TB History and Regimen - 0:00:03.71
• Saving TB History and Regimen - 0:00:03.29

[Mstjamush]

Tests after refactors (Includes cache on forms)

Device Specs

Phone - SAMSUNG A12
RAM - 3GB
ROM - 64GB
Android Version - 12

_WiFi - 20Mbps
Number of Clients - 7584

Results

• Loading the Find person Register - 0:00:08.24
• Search Find Person Register - 0:00:03.17
• Loading Register person form - 0:00:03.94
• Saving a Register person - 0:00:07.22
• Loading Record ART Number form - 0:00:02.56
• Saving a Record ART Number -
• Loading Clinical Stage and CD4 form - 0:00:02.29
• Saving a Clinical Stage and CD4 -
• Loading Welcome Service Newly Diagnosed form - 0:00:02.16
• Saving a Welcome Service Newly Diagnosed -
• Loading TB/Covid Screening - 0:00:02.01
• Saving TB/COVID screening -
• Loading Vitals Page - 0:00:02.94
• Saving Vitals -
• Loading Index Case Testing - 0:00:02.29
• Saving Index Case Testing -
• Loading Guardian Registration - 0:00:02.40
• Saving Guardian Registration -
• Loading TB History and Regimen - 0:00:02.84
• Saving TB History and Regimen -

Save forms has a questionnaire issue

[pld]

was this performed on a version of the SDK with this PR included? https://github.com/google/android-fhir/pull/2565/files

[LZRS]

was this performed on a version of the SDK with this PR included? https://github.com/google/android-fhir/pull/2565/files

No, the sdk versions didn't include the PR...we instead used chunking of the parameters from the app's side to prevent the error

[Mstjamush]

Tests after refactors (Includes cache on forms)

Device Specs

Phone - SAMSUNG A12
RAM - 3GB
ROM - 64GB
Android Version - 12

_WiFi - 20Mbps
Number of Clients - 7584

Results

• Loading the Find person Register - 0:00:40.43
• Search Find Person Register - 0:00:02.87
• Loading Register person form - 0:00:03.65
• Saving a Register person - 0:00:09.24
• Loading Record ART Number form - 0:00:02.27
• Saving a Record ART Number - 0:00:02.75
• Loading Clinical Stage and CD4 form - 0:00:02.49
• Saving a Clinical Stage and CD4 - 0:00:03.53
• Loading Welcome Service Newly Diagnosed form - 0:00:03.34
• Saving a Welcome Service Newly Diagnosed - 0:00:08.11
• Loading TB/Covid Screening - 0:00:02.68
• Saving TB/COVID screening - 0:00:19.89
• Loading Vitals Page - 0:00:08.42
• Saving Vitals - 0:00:03.63
• Loading Index Case Testing - 0:00:03.46
• Saving Index Case Testing - 0:00:02.12
• Loading Guardian Registration - 0:00:03.79
• Saving Guardian Registration - 0:00:02.81
• Loading TB History and Regimen - 0:00:04.02
• Saving TB History and Regimen - 0:00:03.12

@Gental-Giant the issue was fixed

[pld]

Interesting that load time jumped up so much

[f-odhiambo]

Update July 1st 2024

1. To be tested on main to generate new statistics for comparison

[f-odhiambo]

@ndegwamartin Can you provide an update on this ? Aug 5th

CC @Mstjamush

[Mstjamush]

Device Specs

Tecno Spark 7
RAM - 3GB
ROM - 64GB
Android - 11

WiFi - 20Mbps
Number of House Holds - 938


Performance

- Login (after initial sync) - 0:02:24.09
- Add House Hold launch - 0:00:03.75
- Add House Hold save - 0:00:24.70
- Add HH member launch - 0:00:26.85
- Add HH member save - 0:00:33.38
- WASH Check lauch - 0:00:02.70
- WASH Check save - 0:00:10.75
- Client Baseline Form launch - 0:00:04.58
- Client Baseline save - 0:00:09.35
- Child Routine visit form launch - 0:00:03.31
- child routine visit save - 0:00:08.31
- Immunization form launch - 0:00:02.55
- Immunization save -0:00:07.86
- Sick child asessment form launch - 0:00:03.30
- Sick child assessment save - 0:00:09.78
- HIV Pre-screen form launch - 0:00:02.56
- HIV Pre-screen form save - 0:00:06.01
- Tb screening form launch - 0:00:02.98
- TB screening form save - 0:00:11.43

[ndegwamartin]

All the profiling under this ticket are covered so closing this. We can open another ticket once we target a new profiling.