Skip to content
This repository has been archived by the owner on Aug 25, 2018. It is now read-only.

Rebuild: parallelize dependencies downloads #10

Merged
merged 1 commit into from
Dec 5, 2015
Merged

Rebuild: parallelize dependencies downloads #10

merged 1 commit into from
Dec 5, 2015

Conversation

petrkotek
Copy link
Contributor

This PR makes download of dependencies parallel when using gvt rebuild (i.e. addresses #7) and significantly improves speed of the rebuild.

By default, 8 connections are used, however it's possible to override it using:
gvt rebuild --connections X.

This significantly improved speed of our travis builds (gvt rebuild was originally taking 115s, now it's 21s with 8 parallel download connections, i.e. more than 5x faster).

Use ?w=1 to review.

Please let me know, what do you think about it. Thanks!

@petrkotek
Copy link
Contributor Author

I did some more testing of speed - results below

  • Connections = count of concurrent connections
  • Time = time spent with gvt rebuild
Connections Time
1 10m 54s
2 6m 3s
4 3m 38s
8 1m 59s
16 1m 18s
32 1m 8s

Note: there was 53 dependencies.

@petrkotek petrkotek mentioned this pull request Nov 17, 2015
@petrkotek
Copy link
Contributor Author

ping @FiloSottile - i was wondering what's your opinion about this PR?

@FiloSottile
Copy link
Owner

Thanks @petrkotek! Looks good

FiloSottile added a commit that referenced this pull request Dec 5, 2015
Rebuild: parallelize dependencies downloads
@FiloSottile FiloSottile merged commit daff985 into FiloSottile:master Dec 5, 2015
@petrkotek petrkotek deleted the parallel-rebuild branch December 6, 2015 09:07
@petrkotek
Copy link
Contributor Author

Thank you, @FiloSottile 🙇

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants