Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Stream CSV when processing server entities #6353

Merged
merged 2 commits into from
Aug 19, 2024

Conversation

seadowg
Copy link
Member

@seadowg seadowg commented Aug 16, 2024

Closes #6350
Blocked by getodk/javarosa#792

This removes the CSV to XML parsing that was being performed and instead just parses the instance CSV directly as a stream which should improve the memory footprint when downloading entity forms. There may be a slight speed up as well as we don't convert the CSV to an in-mem XML representation, but this will be minor at best.

Why is this the best possible solution? Were any other approaches considered?

Not a lot to discuss here!

How does this change affect users? Describe intentional changes to behavior and behavior that could have accidentally been affected by code changes. In other words, what are the regression risks?

The main risk will be to forms with entities (follow up and update forms). Nothing else should be affected. The only device that predictably experienced OOMs was @lognaturel's Galaxy A13, so let's let her confirm that this fixes that.

Before submitting this PR, please make sure you have:

  • added or modified tests for any new or changed behavior
  • run ./gradlew connectedAndroidTest (or ./gradlew testLab) and confirmed all checks still pass
  • added a comment above any new strings describing it for translators
  • added any new strings with date formatting to DateFormatsTest
  • verified that any code or assets from external sources are properly credited in comments and/or in the about file.
  • verified that any new UI elements use theme colors. UI Components Style guidelines

@seadowg seadowg removed the blocked label Aug 19, 2024
@seadowg seadowg marked this pull request as ready for review August 19, 2024 08:36
@seadowg seadowg requested a review from grzesiek2010 August 19, 2024 08:36
@lognaturel
Copy link
Member

lognaturel commented Aug 19, 2024

I can confirm that this allows me to download and use the 100k Entities Filter form with the Galaxy A13. 👍

@grzesiek2010 grzesiek2010 merged commit 490f750 into getodk:master Aug 19, 2024
6 checks passed
@lognaturel lognaturel mentioned this pull request Aug 19, 2024
6 tasks
@seadowg seadowg deleted the csv-parsing branch August 20, 2024 07:47
@WKobus
Copy link

WKobus commented Aug 20, 2024

Tested with Success

Verified on device with Android 14

Verified cases:

  • Update and registration forms
  • Performance on 100k entity forms

@dbemke
Copy link

dbemke commented Aug 20, 2024

Tested with Success

Verified on device with Android 10

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Reduce memory consumed when parsing Entity List CSV
6 participants