Running out of memory on stack setup

[kitfre]

Please use the following bug reporting template to help produce actionable and reproducible issues. Please try to ensure that the reproduction is minimal so that the team can go through more bugs!

• A brief description
  I installed ghc and stack on build 14986 and attempted to run stack setup.

• Expected results
  This should setup and install the latest version of ghc globally.

• Actual results (with terminal output if applicable)
  stack setup runs out of memory when making ghc. However, only 4 GBs of RAM are being used when I have 8. The output from stack:

strip: createProcess: fork: resource exhausted (Cannot allocate memory)
make[1]: *** [install_packages] Error 1
make: *** [install] Error 2

Error encountered while installing GHC with
  make install
  run in /home/kitfreddura/.stack/programs/x86_64-linux/ghc-8.0.1.temp/ghc-8.0.1/

• Your Windows build number
  14986

• Steps / All commands required to reproduce the error from a brand new installation

sudo apt-get install ghc
curl -sSL https://get.haskellstack.org/ | sh
stack setup

• Required packages and commands to install
  ghc, stack

[benhillis]

I've recently made some changes that change the way that the fork system call determines out of memory conditions. I'll give you repro steps a try and see if things are better on a build with my change (which is not yet released to insiders yet).

@KitFreddura - can you clarify what you mean by However, only 4 GBs of RAM are being used when I have 8

How are you determining how much ram is being used?

[fpqc]

@benhillis Will there be one more build this year?

[benhillis]

@fpqc - Good question, I honestly don't know.

[fpqc]

Time to bug Dona on Twitter =O!

[kitfre]

@benhillis I used task manager to monitor my memory usage while running stack setup, is that an accurate metric?

[benhillis]

Yes that should be accurate. And it was showing 4 / 8 gigs in use?

[kitfre]

Yes, exactly that

[benhillis]

Ok, I have confirmed that I was successfully able to run "stack setup" on a build that has my memory fix. How long does that operation typically take on native Ubuntu? It took quite a long time (over an hour) but that could be because I was using a debug driver which has a lot of extra checks and validation.

Once I've configured the Haskell tool stack do you have a pointer to some unit tests or other commands I can run to validate everything is functional?

[carlpaten]

If you want to smoke-test the hell out of your stack installation, you could try building Pandoc and then running an example or two.

stack setup is usually quite fast, minutes at most.

[benhillis]

Thanks for the pointer. I'll try it again on a non-debug build.

[yuliswe]

When running top, I see all programs are using 0% memory. Possibly the memory usage detection is wrong.
#1384 #1279 #1205 related?

[NickSeagull]

Can confirm what @YLiLarry said, all programs using 0%. Before and after this point.

[benhillis]

The memory usage reporting is unrelated. That's related to procfs files not containing actual memory usage info. That issue is being fixed right now.

[TerrorJack]

I encountered exactly the same problem with 14986. Switching to 15007 solved the problem, however stack setup took approximately 2hr on my surface book, which shouldn't be the case with normal Ubuntu installation. And everything Haskell-related (building packages, launching ghci, etc) is dreadfully slow for now.

[pnf]

As of 15058, I am able to stack setup, though it took somewhere north of 3 hours, finishing sometime overnight.
Is there actually reason to believe that the current performance problems are related to the original memory allocation issue.

[alandao]

I'm getting huge lag when running "stack setup". I left it on for 30 minutes, and while using another terminal, basic command line utilities(ls, cd, git) started taking a very long time(over 10s). Perhaps it is a memory allocation problem..

[benhillis]

@alandao - The performance issue is being tracked by #1671 - @stehufntdev is working on a fix.