Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow users to upload multiple FTP files within a single request #4513

Merged
merged 40 commits into from
Oct 9, 2017

Conversation

guerler
Copy link
Contributor

@guerler guerler commented Aug 29, 2017

This PR adds another tab to the uploader view which allows users to submit multiple FTP files at once. In contrast to the regular upload view, this option does not allow the selection of individual datatypes or genome builds per ftp file. All ftp files are submitted using the same datatype/genome specification. However, the advantage is that all FTP files can be submitted to the upload tool at once, which improves the performance for large sets. Fixes #4347 and #1090.

@guerler guerler added this to the 17.09 milestone Aug 29, 2017
@guerler guerler requested a review from jmchilton August 29, 2017 18:33
@guerler guerler changed the title Ftp uploader update FTP uploader update Aug 29, 2017
@guerler guerler changed the title FTP uploader update Allow users to upload multiple FTP files within a single request Aug 29, 2017
@guerler guerler requested review from martenson and removed request for jmchilton August 29, 2017 19:44
@martenson
Copy link
Member

Do I need to do something else than checkout guerler/ftp_uploader and start Galaxy to test this? Because I cannot see the ftp tab.

@guerler
Copy link
Contributor Author

guerler commented Sep 6, 2017

@martenson thats the correct branch, we embedded the performance improvements into the regular uploader and removed the ftp tab. You could test the performance differences by selecting many ftp files in the regular view and then by clicking on start with and without this PR. However the multifile submission with differing dbkey/datatypes is still in the works.

@martenson
Copy link
Member

martenson commented Sep 6, 2017

That means we are still stuck with the popover-over-modal view into FTP directory. I think we need to improve on this to use actual tree-based folder browser. (And maybe move the whole upload view into mid pane instead of a modal to give it more of a canvas.)

Nevertheless this PR does not seem to be working for me. Nothing gets imported to history.

screenshot 2017-09-06 17 24 30

@guerler
Copy link
Contributor Author

guerler commented Sep 6, 2017

@martenson as I mentioned However the multifile submission with differing dbkey/datatypes is still in the works. All you can test are performance differences in working with large ftp file sets. I let you know when the submission feature for the backend works. See also: #4563.

@martenson
Copy link
Member

@guerler I selected 4 FTP files and nothing got imported into history. All have autodetect and ?. I did not even see a request to upload in the devtools.

@guerler
Copy link
Contributor Author

guerler commented Sep 6, 2017

@martenson that is correct the submission to the backend has to be added. I let you know when we added it.

@guerler guerler modified the milestones: 18.01, 17.09 Sep 11, 2017
@guerler
Copy link
Contributor Author

guerler commented Sep 11, 2017

Requires #4563.

@jmchilton
Copy link
Member

Thanks for moving the milestone @guerler - sorry I wasn't able to get #4563 done in time for the release.

Conflicts:
 static/scripts/bundled/libs.bundled.js
 static/scripts/bundled/libs.bundled.js.map
@jmchilton
Copy link
Member

Very nice - thanks for this!

@jmchilton jmchilton merged commit e03bedf into galaxyproject:dev Oct 9, 2017
@guerler
Copy link
Contributor Author

guerler commented Oct 9, 2017

Thank you. Thanks a lot for #4563 and your overall help with this.

jmchilton added a commit to jmchilton/galaxy that referenced this pull request Oct 9, 2017
@guerler guerler deleted the ftp_uploader branch February 19, 2020 23:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants