Resolving dependencies takes too long

[Stift]

We have around 90+ direct dependencies and like ~200 dependencies in total in our project and since v2.52.9 the resolving takes 'forever'. It seems for me, that the fix of #1520 has caused that problem. Before an paket install took like 7-10s and now we are waiting +15min at least (I had to stop this and did not measure the correct timing).

[forki]

yeah looks like a massive perf issue. I will investigate
/cc @isaacabraham

[forki]

I just reverted that change. Can just please check if perf is restored? Seems I need to investigate deeper

[forki]

pushed 2.52.12 - I think I verschlimmbessert it in 2.52.11

[Stift]

Tested it with 2.52.12. and it's back to normal. Thx.

[forki]

OK now I need to find a proper fix for #1520
@Stift I guess I need to push another version soon and ask you again if perf is still good ;-)

[Stift]

Just do so. I'm glad to test it.
But today is St. Patricks Day and I'm off and out in a few hours:

[forki]

2.52.13 will be online in 2min. Please give it a try

[Stift]

that will fit

[forki]

so it's online

[Stift]

I'm already testing it. I get some warnings that I cannot specifcally say what they are about:

Could not resolve package Npgsql:
 - Dependencies file requested: 2.2.5-custom
 - <customPackage1> 1.0.11044 requested: >= 0
 - <customPackage2> 1.0.11149 requested: >= 2.2.5
 - <customPackage3> 1.1.10087 requested: >= 2.0.0 < 3.0.0

Also some of the numbers do not fit to the package itself. Is something mixing up here?

[forki]

shit. soo I'm not done

[Stift]

Also after a second run of install downloads now all packages again and this again takes a while.
First run: 26s
Second run (downloading?): +5min

[Stift]

sorry for that

[forki]

ok this one is really driving me nuts.

I released another version. Can you give me your deps file by any chance?

[Stift]

I'll send it to you, but it contains many of our own packages and we use also a Klondike server as backend. i'll send it to you via mail.

[forki]

i guess this means 2.52.14 is still broken...

[Stift]

It has the same strange warning as before, but this time it does not tries to redownload.

[Stift]

I have sent you our dep file per mail.

[forki]

just trying to reproduce with your deps file, but I assume in order to make it really hard the conflict is only in your custome nugets ;-)

[Stift]

no worries. I can pin paket for now. For today I'm out. I have some assumption on that as well. Before monday I will not get into that. Would be glad if you can give me a hint what I should look at. Lately I saw on the verbose level that one package was repeatedly tried to "retrieve information".

[forki]

so the public part of your deps file resolves without error. I guess we need to look into this next week.

[tsibelman]

@forki what do you think to separate releases into stable and beta channels because these frequent changes with high potential for regression, are affecting people day to day work.

[forki]

we usally do that, but the v3 branch is blocking the alpha channel.
I aslo starting to add property based test around this part.

[tsibelman]

Maybe you need 3 channels, stable, alpha for bigger longer changes and beta for smaller one

[Stift]

Ok. I think I found the error in our package that was causing the warning:

Could not resolve package Npgsql:
 - Dependencies file requested: 2.2.5-custom
 - <customPackage1> 1.0.11044 requested: >= 0
 - <customPackage2> 1.0.11149 requested: >= 2.2.5
 - <customPackage3> 1.1.10087 requested: >= 2.0.0 < 3.0.0

The <customPackage2> had its dependency too strict and therefore the Npgsql.2.2.5-custom was not fitting in its constraint. It was obvious after I saw it. The first time I misread the warning and thought that the customPackages were requested, but this morning I just discovered that it was meant in a way that this package requested npgsql of this version.

[forki]

So what paket says is correct?

[Stift]

Yes. I misunderstood the message! Sorry for that confusion.

If I read the message like this, I would maybe understand this earlier.
Could not resolve package Npgsql:
- Dependencies file requested package Npgsql: 2.2.5-custom
- 1.0.11044 requested package Npgsql: >= 0
- 1.0.11149 requested package Npgsql: >= 2.2.5
- 1.1.10087 requested package Npgsql: >= 2.0.0 < 3.0.0

Should I make a pr for this?

[forki]

Yes everything that improves error messages is good.
On Mar 18, 2016 18:03, "Christian Fiebrig" [email protected] wrote:

Yes. I misunderstood the message! Sorry for that confusion.

If I read the message like this, I would maybe understand this earlier.
Could not resolve package Npgsql:

• Dependencies file requested package Npgsql: 2.2.5-custom
• 1.0.11044 requested package Npgsql: >= 0
• 1.0.11149 requested package Npgsql: >= 2.2.5
• 1.1.10087 requested package Npgsql: >= 2.0.0 < 3.0.0

Should I make a pr for this?

—
You are receiving this because you were mentioned.
Reply to this email directly or view it on GitHub
#1524 (comment)

[forki]

@Stift seems that I need your help again. Property based testing revealed another issue around that code. I was able to fix that but I want to know if how bad the perf is. Can you please check 3.0.0-alpha079? Thanks mate

[Stift]

No problem.
stats for paket install

Commit	v2.54.0	v3.0.0-alpha079
Unclean (with error in nupkg)	24s	2h+
Clean (paket.lock is fine)	8s	8s
Clean (paket.lock has to be updated)	11s	13s

and for a paket update nuget package only on a clean commit

v2.54.0	v3.0.0-alpha079
161s	161s

The unclean commit was the one that originally made problems and where we had a package which was not configured well.

I mean the only issue that I see is that with this unclean package we get into a situation where paket does not behave that well. When I looked in the verbose output of 2.52.10 (where the original problem started) I had the feeling paket was running in circles.

[forki]

So apart from a strange state it was ok?

[Stift]

yes. Seems like current stable and alpha are similiar.

[Stift]

I just fear a bad user experience when paket cannot detect that it runs in a regression.

[forki]

Yes we need to find that error. Question is how to reproduce on my machine.
On Mar 22, 2016 8:56 PM, "Christian Fiebrig" [email protected]
wrote:

I just fear a bad user experience when paket cannot detect that it runs in
a regression.

—
You are receiving this because you modified the open/close state.
Reply to this email directly or view it on GitHub
#1524 (comment)

[forki]

ok I added another 3.0alpha package. I hope it solves that 2h hour thing

[Stift]

I tested with 3.0.0-alpha80 and with the unclean package paket install takes about 27 seconds. On a clean repo without the failing package we still have around 15 seconds. For me this looks definetely better and acceptable (from a user perspective).

Update is also fine on the clean one.

[forki]

thanks buddy.

[Stift]

no problem. Glad to serve and give something back.