build --profile fails with cannot find object file that lifts template haskell when no library

[philderbeast]

General summary

I can build with stack build but with stack build --profile the build fails with
cannot find object file EmbedString.o when all source files are for the executable. If instead, I have the EmbedString module in a library, the build succeeds with and without --profile.

I'm following the docs ...

To profile a component of the current project, simply pass the --profile flag to stack.

Steps to reproduce

For example:

1. Start a new project with stack new simple
2. Replace Lib with EmbedStr

module EmbedString (embedStr) where

import Language.Haskell.TH
import Language.Haskell.TH.Syntax (lift)

embedStr :: IO String -> ExpQ
embedStr readStr = lift =<< runIO readStr

3. Add template-haskell to the library build-depends and and raw-strings-qq to the executable build-depends.
4. Update app/Main.hs to the following ...

{-# LANGUAGE TemplateHaskell #-}
{-# LANGUAGE QuasiQuotes #-}

module Main where

import EmbedString (embedStr)
import Text.RawString.QQ (r)

license :: String
license = intro ++ lic ++ newline
    where
        lic= $(embedStr (readFile "./LICENSE"))
        newline = [r|
|]
        intro = [r|
Included Software License
-------------------------
The LICENSE for this app is ...
|]

main :: IO ()
main = putStr license

4. Run command stack build --profile. This should work.
5. Get rid of the library in the cabal file, add src to hs-source-dirs and template-haskell to build-depends.
6. Run command stack build --profile fails but stack build succeeds.

For the reproduction steps see stack-profile-embed-string.

The change that introduces the problem, dropping the library.

Expected

That the two commands stack build and stack build --profile either both succeed or both fail.

Actual

I can build with stack build but with stack build --profile the build fails with
cannot find object file EmbedString.o.

E:\Dev\Src\temp\stack-profile-embed-string [master ≡]> stack clean
E:\Dev\Src\temp\stack-profile-embed-string [master ≡]> stack build
Warning: File listed in simple\simple.cabal file does not exist: README.md
simple-0.1.0.0: configure (exe)
Configuring simple-0.1.0.0...
simple-0.1.0.0: build (exe)
Preprocessing executable 'simple-exe' for simple-0.1.0.0...
[1 of 2] Compiling EmbedString      ( src\EmbedString.hs, .stack-work\dist\b7fec021\build\simple-exe\simple-exe-tmp\EmbedString.o )
[2 of 2] Compiling Main             ( app\Main.hs, .stack-work\dist\b7fec021\build\simple-exe\simple-exe-tmp\Main.o )
Linking .stack-work\dist\b7fec021\build\simple-exe\simple-exe.exe ...
Warning: File listed in simple\simple.cabal file does not exist: README.md
simple-0.1.0.0: copy/register
Installing executable(s) in
E:\Dev\Src\temp\stack-profile-embed-string\.stack-work\install\3122d1ef\bin
E:\Dev\Src\temp\stack-profile-embed-string [master ≡]> stack clean
E:\Dev\Src\temp\stack-profile-embed-string [master ≡]> stack build --profile
Warning: File listed in simple\simple.cabal file does not exist: README.md
simple-0.1.0.0: configure (exe)
Configuring simple-0.1.0.0...
simple-0.1.0.0: build (exe)
Preprocessing executable 'simple-exe' for simple-0.1.0.0...
[1 of 2] Compiling EmbedString      ( src\EmbedString.hs, .stack-work\dist\b7fec021\build\simple-exe\simple-exe-tmp\EmbedString.p_o )
[2 of 2] Compiling Main             ( app\Main.hs, .stack-work\dist\b7fec021\build\simple-exe\simple-exe-tmp\Main.p_o )
E:\Dev\Src\temp\stack-profile-embed-string\simple\app\Main.hs:12:14: fatal:
    cannot find object file `.stack-work\dist\b7fec021\build\simple-exe\simple-exe-tmp\EmbedString.o'
    while linking an interpreted expression
Warning: File listed in simple\simple.cabal file does not exist: README.md

--  While building package simple-0.1.0.0 using:
      C:\sr\setup-exe-cache\x86_64-windows\Cabal-simple_Z6RU0evB_1.24.0.0_ghc-8.0.1.exe --builddir=.stack-work\dist\b7fec021 build exe:simple-exe --ghc-options " -ddump-hi -ddump-t
o-file"
    Process exited with code: ExitFailure 1

The earlier version with the library in the cabal file doesn't have this problem ...

E:\Dev\Src\temp\stack-profile-embed-string [master ≡]>git checkout 75207b5558212764429e066c14bb5cc911a739b1
E:\Dev\Src\temp\stack-profile-embed-string [(75207b5...)]> stack clean
E:\Dev\Src\temp\stack-profile-embed-string [(75207b5...)]> stack build --profile
Warning: File listed in simple\simple.cabal file does not exist: README.md
simple-0.1.0.0: configure (lib + exe)
Configuring simple-0.1.0.0...
simple-0.1.0.0: build (lib + exe)
Preprocessing library simple-0.1.0.0...
[1 of 1] Compiling EmbedString      ( src\EmbedString.hs, .stack-work\dist\b7fec021\build\EmbedString.o )
[1 of 1] Compiling EmbedString      ( src\EmbedString.hs, .stack-work\dist\b7fec021\build\EmbedString.p_o )
Preprocessing executable 'simple-exe' for simple-0.1.0.0...
[1 of 1] Compiling Main             ( app\Main.hs, .stack-work\dist\b7fec021\build\simple-exe\simple-exe-tmp\Main.p_o )
Linking .stack-work\dist\b7fec021\build\simple-exe\simple-exe.exe ...
Warning: File listed in simple\simple.cabal file does not exist: README.md
simple-0.1.0.0: copy/register
Installing library in
E:\Dev\Src\temp\stack-profile-embed-string\.stack-work\install\3122d1ef\lib\x86_64-windows-ghc-8.0.1\simple-0.1.0.0-9cmvvgP5TyOE3ldSwmzx1G
Installing executable(s) in
E:\Dev\Src\temp\stack-profile-embed-string\.stack-work\install\3122d1ef\bin
Registering simple-0.1.0.0...

Stack version

E:\Dev\Src\temp\stack-profile-embed-string [master ≡]> stack --version
Version 1.3.2, Git revision 3f675146590da4f3edf768b89355f798229da2a5 x86_64 hpack-0.15.0

Method of installation

Official binary, downloaded from stackage.org followed by stack upgrade.

[philderbeast]

With the EmbedString module in the library both EmbedString.o and EmbedString.p_o are built but with that module in the executable, only EmbedString.p_o gets built.

[mgsloan]

Thanks for the report! I can indeed reproduce the issue. Not sure what's going on there.

A bit of googling turns up this old cabal issue - haskell/cabal#1207 . While likely not exactly the same issue, it does seem likely that it's an upstream issue with cabal, as we delegate building to it.

[lehins]

GHC documentation talks about this issue:

7.17.5. Using Template Haskell with Profiling
Template Haskell relies on GHC's built-in bytecode compiler and interpreter to run the splice expressions. The bytecode interpreter runs the compiled expression on top of the same runtime on which GHC itself is running; this means that the compiled code referred to by the interpreted expression must be compatible with this runtime, and in particular this means that object code that is compiled for profiling cannot be loaded and used by a splice expression, because profiled object code is only compatible with the profiling version of the runtime.
This causes difficulties if you have a multi-module program containing Template Haskell code and you need to compile it for profiling, because GHC cannot load the profiled object code and use it when executing the splices. Fortunately GHC provides a workaround. The basic idea is to compile the program twice:

1. Compile the program or library first the normal way, without -prof.
2. Then compile it again with -prof, and additionally use -osuf p_o to name the object files differently (you can choose any suffix that isn't the normal object suffix here). GHC will automatically load the object files built in the first step when executing splice expressions. If you omit the -osuf flag when building with -prof and Template Haskell is used, GHC will emit an error message.

So here is a manual workaround for this issue:

$ stack build
$ rm .stack-work/dist/x86_64-linux/Cabal-1.24.0.0/stack-build-cache
$ stack build  --ghc-options="-osuf p_o" --profile
$ stack exec -- simple-exe +RTS -p

[mgsloan]

Interesting! Perhaps stack should always do such a double build? Could make compilation take too long for the common case.

[flip111]

I've run into the same issue when the .stack-work folder is not present and i want to build my project with profiling right away.

Perhaps stack should always do such a double build?

Maybe there is a way to detect that a normal build has already been performed for the relevant code in question. Maybe it's possible to just compile the module without profiling that is causing this issue with template haskell. Then after this all modules can be build with profiling.

[cxandru]

Edit: I fixed the issue in my case by setting -rtsopts as ghc-options in my package.yaml.

Also have this issue. I get: Cannot find Object file .stack-work/dist/x86_64-linux/Cabal-2.2.0.1/build/main/main-tmp/Commons.dyn_o, where Commons is a project file using some TH-based lens package functions.
If I run stack build first, then stack build --profile, I get 'Most RTS options are disabled. Link with -rtsopts to enable them.'

[snoyberg]

I've investigated this a bit:

• raw-strings-qq is a red herring, removing it seems to still cause the build to fail
• There's no "two stage" necessary to trigger this bug. Simply starting with a clean directory with no library also causes it to fail
• This works with cabal-install, so there's got to be some combination of flags we can pass that will get this to work
• I tried adding --disable-executable-dynamic to the configure call, and that didn't fix things

I am able to reproduce this directly against the Cabal library, however.

1. ghc Setup.hs
2. ./Setup configure --enable-profiling
3. ./Setup build fails, just like stack build --profile

Going further down the rabbit hole, I see in the cabal-install log that --disable-profiling is enabled. Running the executable generated by cabal-install with +RTS -p indicates that the executable does not support profiling. So this may in fact be unsupported for some reason. Still investigating...

[snoyberg]

Alright, this appears to be an upstream issue, either in Cabal or GHC. Closing in favor of haskell/cabal#5961

[snoyberg]

And I remembered the resolution for this problem: can you try adding other-extensions: TemplateHaskell to the appropriate stanza and see if that fixes things?