dep importing from godep produces 30x bigger vendor folder and takes 10 minutes to do so

[dlsniper]

In the following project https://github.com/gopheracademy/gopher , running dep init needs more than 10 minutes to work and produces a vendor/ folder that's 106MB compared to the 3MB one produced by godep. It also vendors packages that are not originally present in the godep manifest nor are needed by the project to compile.
Since this is highly reproducible, just download gopher and run dep init to obtain the same results.

dep version: f134697
gopher version: 7bb1dd8a82cff5fb8172bcb768d0decd9ba80420

[AlekSi]

running dep init needs more than 10 minutes

For me, it takes 50 seconds with a clean cache.

It also vendors packages that are not originally present in the godep manifest nor are needed by the project to compile.

That's the reason for 106MB – some versions are off. Specifically, google.golang.org/api – it grew a lot. And a reason for that, I guess, is that Godeps.json contains packages and import paths, while dep mostly tracks repositories.

[dlsniper]

For me, it takes 50 seconds with a clean cache.

I don't know where the cache is nor should I need to know where it is for the tool to work quickly. And if the purpose of the cache is to speed things up but it slows them down then I guess that's a valid problem on its own.

That's the reason for 106MB – some versions are off.

Which I've specified as a separate problem in #1070. That in itself is a huge problem.

And a reason for that, I guess, is that Godeps.json contains packages and import paths, while dep mostly tracks repositories.

I believe so, but also dep does vendor dependencies that are not used nor declared in Godeps.json. That also contributes to the increase in storage. Normally it shouldn't be a problem, but it does reflect the regression compared to existing tools.

[AlekSi]

I don't know where the cache is nor should I need to know where it is for the tool to work quickly.

I'm just saying that timing is not "highly reproducible".

[sdboyer]

ok, a few things here!

produces a vendor/ folder that's 106MB compared to the 3MB one produced by godep

some of this is likely a product of the version changing that's happening with google.golang.org/api - more deps are coming in. that's the most likely explanation for this:

I believe so, but also dep does vendor dependencies that are not used nor declared in Godeps.json.

the newer version of that project, that the solver picked for whatever reason - brought in new dependencies, which means more things end up in Gopkg.lock.

but yes, that's only part of the explanation. because:

It also vendors packages that are not originally present in the godep manifest nor are needed by the project to compile.

you'll want to run dep prune afterwards - that'll clean out unused packages. we're working to make that implicit in #944.

needs more than 10 minutes to work

performance in general is something we've prioritized below correctness and safety, which is why it's taking a little while to get to. however, we're making active progress on two things that'll reduce running times significantly. the FAQ has more info.

[sdboyer]

there's nothing original in this issue that isn't already covered by others.