Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Simplify exports #1864

Closed
wants to merge 3 commits into from
Closed

Conversation

jeffm2001
Copy link
Collaborator

Description

This attempts to fix out-of-memory errors and timeouts when exporting larger (100-200k) campaigns. I have not tested it with really large campaigns.

What this PR does is simplify the collection of data in the export code. Previously, exportCampaign looped through all assignments in the campaign and then looped through each row in the assignment — potentially resulting in hundreds of thousands of queries for one export.

This changes it so that data is pulled with a few big queries (contacts, messages, question responses, tags), and does one loop through to combine/format the results in-place.

Checklist:

  • I have manually tested my changes on desktop and mobile
  • The test suite passes locally with my changes
  • If my change is a UI change, I have attached a screenshot to the description section of this pull request
  • My change is 300 lines of code or less, or has a documented reason in the description why it’s longer
  • I have made any necessary changes to the documentation
  • I have added tests that prove my fix is effective or that my feature works
  • My PR is labeled [WIP] if it is in progress

Copy link
Collaborator

@schuyler1d schuyler1d left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bless you for this.

@schuyler1d
Copy link
Collaborator

This was successfully merged for Spoke 10.0

@schuyler1d schuyler1d closed this Feb 8, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants