Building with stack on Travis CI - dependency compilation fails sometimes

[rsasse]

We are using stack to build the https://github.com/tamarin-prover/tamarin-prover cryptographic protocol verification tool. As part of our tests we run a build on Travis CI. Builds work perfect locally, but on Travis only work some of the time. Other times, the compilation fails on some dependency, usually the mwc-random package.

For a working build's log see:
https://travis-ci.org/tamarin-prover/tamarin-prover/builds/77341662
For a failing build's log see:
https://travis-ci.org/tamarin-prover/tamarin-prover/builds/77350525

It seems that maybe the order in which the packages build does matter, but that is just speculation.

Adding further detail, we are installing stack on Travis using

wget -q -O- https://s3.amazonaws.com/download.fpcomplete.com/ubuntu/fpco.key | sudo apt-key add -
echo 'deb http://download.fpcomplete.com/ubuntu/trusty stable main'|sudo tee /etc/apt/sources.list.d/fpco.list
sudo apt-get update && sudo apt-get install stack -y

which works fine. Note that we are using the lts-2.22 resolver so we use GHC 7.8, as we do not support GHC 7.10 (with lts-3.0) due to other issues.

We also tried using the "cache" feature on Travis, hoping it would compile the dependencies once and for all, but apparently that does not work in combination with building via stack. At least adding

cache:
directories:

• $HOME/.stack
• .stack-work

to the .travis.yml file had no effect, everything gets compiled from scratch again.

If there is any further information required from me that could help, please let me know.

[snoyberg]

The problem is at the end with the error message:

Process exited with code: ExitFailure (-9)

A negative exit code means failure due to a signal, and signal 9 is OOM (out of memory). The problem is that you're running out of memory on Travis. It's nondeterministic since it depends on which libraries are being built at the same time.

One option is to build the biggest memory users (SHA and mwc-random in this case it seems) in a pre-build step, with concurrent builds turned off:

stack build -j 1 mwc-random SHA

I would recommend using a cached build. In order to do so, you'll need some newer Travis tricks. There's a Wiki page devoted to this: https://github.com/commercialhaskell/stack/wiki/Travis. As it happens, I just finished writing up one such Travis file an hour ago: https://github.com/commercialhaskell/all-cabal-hashes-tool/blob/master/.travis.yml.

[rsasse]

Thanks much! Trying this out right now and will let you know what happens.

[rsasse]

Pre-building the two biggest memory users solved the problem. Caching also worked and speeds up our builds tremendously.

Thank you very much again for your super quick and very helpful answer!