Add multithreading to Dolos file download #51
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
What?
Speed up report sending to Dolos by doing simple multithreading. This PR also improves the UX by providing a progress indicator once the user clicks the "View in Dolos" button alongside a description of Dolos and it's current status as a WIP tool.
Why?
Downloading submission files file by file is very slow when there are thousands of submissions. This speeds the process up substantially
How?
Using the concurrent.futures feature in python
Fixes #
Testing
Remember to add or update unit tests for new features and changes.
What type of test did you run?
[ADD A DESCRIPTION ABOUT WHAT YOU TESTED MANUALLY]
Did you test the changes in
Think of what is affected by these changes and could become broken
Translation
Programming style
Have you updated the README or other relevant documentation?
Is it Done?
Clean up your git commit history before submitting the pull request!