Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: sort responses for csv download according to creation date #2028

Merged
merged 3 commits into from
Jun 8, 2021

Conversation

chowyiyin
Copy link
Contributor

@chowyiyin chowyiyin commented May 31, 2021

Problem

Currently, when a CSV of responses are downloaded (in storage mode), the responses are not sorted according to date. Refer to issue #1721

Closes #1721

Solution

The naive approach was adopted to solve this problem. Specifically Array.prototype.sort() was used to sort the responses. This solution seems satisfactory even for a large number of responses. When tested with 100,000 responses, the sorting duration ranged from 1 to 2 seconds. With approximately 500,000 responses, the sorting duration ranged from 9 to 10 seconds, which is not significant compared to the decryption duration of approximately 3 minutes. Throughout the download, the following modal is displayed and no obvious freezes or slowdowns in the spinner were observed. The short clip below shows the condition of the spinner before and during sorting.

csv_download.mov

Tests

  • Submit responses to storage mode form and download CSV. Ensure downloaded CSV is sorted (oldest to newest)

@chowyiyin chowyiyin marked this pull request as ready for review May 31, 2021 08:40
Copy link
Contributor

@liangyuanruo liangyuanruo left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As commented. It might also make sense to decouple the code into a standalone, mutable sort() instance method, instead of embedding it in process().

@chowyiyin chowyiyin requested a review from liangyuanruo June 1, 2021 02:55
Copy link
Contributor

@mantariksh mantariksh left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

also to do the following:

  • make a more conscious decision about what the upper limit of testing should be, and test on staging with this upper limit. 500,000 is a lot but if we expect that some users may download even more, we should ensure that the implementation is stable.
  • test on staging with multiple browsers

@liangyuanruo
Copy link
Contributor

  • test on staging with multiple browsers

I think we can get away with testing on localhost, since code changes are client-side

@mantariksh
Copy link
Contributor

I think we can get away with testing on localhost, since code changes are client-side

I'm not sure about this - doesn't machine performance play a factor? so we'd need to test on Chrome on GSIB before being sure that this works?

@liangyuanruo
Copy link
Contributor

Sure, we can test on GSIB to be safe.

@mantariksh
Copy link
Contributor

to populate a test form with 500k submissions and test performance on GSIB

@liangyuanruo liangyuanruo merged commit 31ed784 into develop Jun 8, 2021
@liangyuanruo liangyuanruo deleted the feat/csvResponseSort branch June 8, 2021 02:25
@karrui karrui mentioned this pull request Jun 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Investigate: sort responses by date (oldest to newest) before download
3 participants