stack setup on appveyor fails

[ndmitchell]

See https://ci.appveyor.com/project/ndmitchell/hlint/build/1.0.44, which just gets stack.exe and runs stack setup --verbose. It fails at the end with:

Running C:\Program Files\Git\cmd\git.exe fetch --tags --depth=1 exited with ExitFailure 128
fatal: Not a git repository (or any of the parent directories): .git
Command exited with code 1

Is it possible on a never-used-before machine stack setup fails?

[borsboom]

I see this in the log:

Restoring build cache
Restoring "c:\sr"...OK

I wonder if something might have gone funny in that build cache. Stack seems to think it already has a package index and just needs to update it, but git doesn't think it's a valid repo.

Could add a check to Stack to ensure the package index is a valid repo before trying to update it (if it's not, just blow it away and start over like it wasn't there), I suppose.

[ndmitchell]

Good thinking, I turned off the cache, and it seems a lot happier. Seems like Stack wiping invalid repos would be very useful.

[borsboom]

A pull request would be most welcome!

[ndmitchell]

For info, I've now seen this on a different project on Appveyor, so not a one-off, but still not common.

[quyse]

Note Appveyor docs says that cache size after compression must be no bigger than 100 Mb. What I see is that on my machine the hackage index alone is > 300 Mb.

[ndmitchell]

@quyse - my guess is that is a wild guideline, and that they don't enforce it, partly because they don't give any hint how big a projects cache is or let you see what it contains.

Looking at the cache sizes for the compiled stuff on my machine I see things in the range 40Mb-160Mb, where the 160Mb compresses down to 20Mb. I'm not sure if I cache anything else or not - it's really pretty hard to figure out in Appveyor.

[quyse]

@ndmitchell Well, you had setting cache: "c:\\sr", i.e. stack root. The biggest thing I've seen there is Hackage index (indices\Hackage\00-index.tar), and I guess it appears when you use non-stackage dependencies. On my machine they took 434 Mb.

I just came up with a script using additional git repository on Gitlab (because they don't have 1Gb per repo/100 Mb per file limits as Github has) in order to cache stack's stuff manually. Script gist: https://gist.github.com/quyse/7178ec5737235bdf60bf, build on appveyor using it: https://ci.appveyor.com/project/quyse/flaw/build/1.0.15. I think thanks to this script I won additional ~20 minutes so build doesn't timeout. It seems like a bit of abuse of Gitlab though (which delivers ~500 Mb for each build), so not really elegant solution. Appveyor really needs bigger cache limits.

[ndmitchell]

My builds take near the 8 min mark, see https://ci.appveyor.com/project/ndmitchell/hlint. I guess I only have Stackage dependencies. I also run stack setup each time and don't cache that bit - caching that adds 20 mins or so.

[ndmitchell]

I'm now seeing this almost every other build for one of my projects, which I does not seem special in any way. There may be some restoring of old states going on that recorrupts things.

[ndmitchell]

PR now in #1580. In the absence of other evidence, I'm going to assume someone at Stack HQ sabotaged my Appveyor instance to make it recur enough to be worth fixing 😛

[ndmitchell]

I think it's reasonable to assume #1580 fixed it, and I'll reopen if that turns out to not be the case.

[ndmitchell]

In the last week I've suddenly got this on 5 different Appveyor repos. They are all using:

Version 1.1.0, Git revision 0e9430aad55841b5ff2c6c2851f0548c16bce7cf (3540 commits) i386 hpack-0.13.0

Seems like a regression in the latest release?

[ndmitchell]

Output is:

Updating package index Hackage (mirrored at https://github.com/commercialhaskell/all-cabal-hashes.git) ...
Fetching package index ...
fatal: Not a git repository (or any of the parent directories): .git
Process exited with ExitFailure 128: C:\Program Files\Git\cmd\git.exe fetch --tags --depth=1
Command exited with code 1

The related code is https://github.com/commercialhaskell/stack/blob/master/src/Stack/PackageIndex.hs#L260. I think the immediate cause is that the caller has switched from using readProcessNull which throws ReadProcessException to callProcessInheritStderrStdout which throws ExitFailure.

I note that at #1580 (comment) I said:

the process functions ... are all a bit weirdly inconsistent

This bug seems to be a symptom of that, with seemingly harmless refactorings changing subtle things like which exceptions are raised.

[ndmitchell]

I've now hit this about 6 times in the last few days. Getting increasingly annoying. Can someone say if they agree/disagree with my diagnosis. I suspect this was broken by 967c3f0 as part of #1223.

[mgsloan]

@ndmitchell Sounds about right. Does e52b2c5 fix it?

Feel free to re-open if not

[ndmitchell]

Thanks @mgsloan! It seems to have stopped recurring in the last couple of days, most appreciated.