Compilation not terminating due to giant .dump-hi (4gb) file when using type families

[haitlahcen]

Hello guys!

Related fixed issue: #4027
Related GHC issue: https://ghc.haskell.org/trac/ghc/ticket/8095#comment:58

My team is experiencing issues when compiling a project that extensively use type families (type level sql with https://github.com/morphismtech/squeal).
We found that the big trouble comes from GHC trying to dump a text file when compiling each unit.
Here are the investigations:

• Stack is asking GHC to dump the .hi file as pretty printed text (see https://stackoverflow.com/a/46399538).
• GHC seems to unfold the types regardless of their size (seems like GHC unfolding thresholds options does not apply to types, see GHC ticket).

Our context:

• We moved our plain SQL queries+migrations to type level using Squeal (involving types families a lot)
• GHC unfolding types resulting in very very long types in the .hi file
• Stack asking GHC to dump the interface, resulting in writing a 4gb .txt (head of the file https://gist.githubusercontent.com/haitlahcen/3f1e83cdc78d48ed2100d51a0ea05844/raw/c49218be742cd3d2450a8f13683a04339b1ab9e7/hi.txt) for each module that import the type level queries.

My questions:

• Could I PR stack to directly read the binary format ? (this would improve compilation performance a lot, relying only on the GHC .hi file). I've already an implementation that can read the dependencies/templatehaskell in the .hi file for ghc7-863. The drawback would be that we would have to support the protocol for each new release of GHC.
• Could we have an option to avoid the dump ?

Current solution:

• Use the -fomit-interface-pragmas that avoid ALL unfolding. This is not very good because we lose inlining.

Thanks for reading.

[mihaimaruseac]

I think adding a flag to avoid the dump or to make the dump use the binary format instead of the plaintext one would be the best.

[mgsloan]

This seems like a really good idea, in favor of merging #4545 once it's been reviewed. I think this should make it so that ghc doesn't need to pass -ddump-hi -ddump-to-file. In some cases it can be rather puzzling to users when various ddump options aren't displayed