--profile and -dynamic builds fail #1691

blitzcode · 2016-01-21T17:36:00Z

I frequently use -dynamic in my GHC options to reduce link times during development. Profile builds seem to first rebuild their dependencies properly, but then fail to find them during compilation of the actual project. It seems like only static versions of the profile libraries are build.

Here's a project of mine with quite a few dependencies and -dynamic set in the Cabal file:

[~] stack --version
Version 1.0.0 x86_64

[~] git clone [email protected]:blitzcode/ray-marching-distance-fields.git
...
[~] cd ray-marching-distance-fields/
[~/ray-marching-distance-fields] stack build --profile
ansi-terminal-0.6.2.3: copying precompiled package
base-orphans-0.4.5: configure
base-orphans-0.4.5: build
ObjectName-1.1.0.1: configure
ObjectName-1.1.0.1: build
...
rmdf-0.1.0.0: configure
Configuring rmdf-0.1.0.0...
rmdf-0.1.0.0: build
Preprocessing executable 'rmdf' for rmdf-0.1.0.0...

/Users/Tim/ray-marching-distance-fields/App.hs:16:8:
    Could not find module ‘Control.Monad.Reader’
    Perhaps you haven't installed the "p_dyn" libraries for package                ‘mtl-2.2.1@mtl_Aue4leSeVkpKLsfHIV51E8’?
    Use -v to see a list of the files searched for.

/Users/Tim/ray-marching-distance-fields/App.hs:17:8:
    Could not find module ‘Control.Monad.State’
    Perhaps you haven't installed the "p_dyn" libraries for package ‘mtl-2.2.1@mtl_Aue4leSeVkpKLsfHIV51E8’?
    Use -v to see a list of the files searched for.

/Users/Tim/ray-marching-distance-fields/App.hs:18:8:
    Could not find module ‘Control.Monad.STM’
    Perhaps you haven't installed the "p_dyn" libraries for package ‘stm-2.4.4.1@stm_EyxFXFfnfY05IrjGaqCxeD’?
    Use -v to see a list of the files searched for.

...

The text was updated successfully, but these errors were encountered:

sjakobi · 2016-04-25T11:52:40Z

I'm seeing a similar error when building the stack project with --profile:

~/s/stack (master) $ stack clean
~/s/stack (master) $ stack build --profile
stack-1.0.5: unregistering (local file changes: src/Control/Concurrent/Execute.hs src/Data/Aeson/Extended.hs src/Data/Attoparsec/Args.hs src/Data...)
stack-1.0.5: configure
Configuring stack-1.0.5...
stack-1.0.5: build
Preprocessing library stack-1.0.5...
[ 1 of 86] Compiling Stack.FileWatch  ( src/Stack/FileWatch.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/Stack/FileWatch.o )
...
[86 of 86] Compiling Stack.SDist      ( src/Stack/SDist.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/Stack/SDist.p_o )
In-place registering stack-1.0.5...
Preprocessing executable 'stack' for stack-1.0.5...
[1 of 2] Compiling Paths_stack      ( .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/autogen/Paths_stack.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Paths_stack.p_o )
[2 of 2] Compiling Main             ( src/main/Main.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Main.p_o )
/home/simon/src/stack/src/main/Main.hs:113:10:
    cannot find normal object file ‘.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Paths_stack.dyn_o’
    while linking an interpreted expression

--  While building package stack-1.0.5 using:
      /home/simon/.stack/setup-exe-cache/x86_64-linux/setup-Simple-Cabal-1.22.7.0-ghc-7.10.3 --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.7.0 build lib:stack exe:stack --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1

There's no dynamic flag involved here AFAIK.

~/s/stack (master) $ stack --version
Version 1.0.5, Git revision 57eb614df50826f38b923e1892cda904288a8ec9 (dirty) (3489 commits) x86_64 hpack-0.12.0

Is there any reason to believe that this is a GHC or Cabal issue or is this likely a bug in stack itself?

mgsloan · 2016-04-25T22:08:28Z

Hmm, that's very strange! I have a similar stack version, same ghc, same Cabal, but it doesn't repro. It's a long shot, but is it solved by the most recent HEAD?

It'd be helpful to have a dockerfile repo + ideally an uploaded container

sjakobi · 2016-04-26T08:54:07Z

Hmm, that's very strange! I have a similar stack version, same ghc, same Cabal, but it doesn't repro. It's a long shot, but is it solved by the most recent HEAD?

57eb614 is the latest HEAD.

It'd be helpful to have a dockerfile repo + ideally an uploaded container

I'll give this a try in the next days.

sjakobi · 2016-04-26T10:23:40Z

I've also reproduced this locally with stack-1.0.4.3 and stack-1.0.2.

Using stack-HEAD and stack-7.8.yaml, i.e. GHC-7.8.4 and Cabal-1.18.1.5, leads to the same error.

sjakobi · 2016-04-27T02:13:54Z

I've run into another instance of this bug, this time closer to what @blitzcode experienced:

CLI output here
Builds logs just in case here

In this case the missing profiling libraries are supposed to come from a procompiled package! I think this might be a hint at the root of the problem.

BTW, during this run I had set apply-ghc-options to locals in the stack.yaml. Because of this I was a bit surprised that the snapshot libraries had to be recompiled at all. Is this a misunderstanding on the intended behaviour of --profile on my part?

$ stack --version
Version 1.0.4.3, Git revision db1afb275ebd64016e8ef0db122f0f03da9e5906 (3242 commits) x86_64

mgsloan · 2016-04-27T03:01:40Z

IIRC, --profile builds still require compilation of all dependencies with profiling, regardless of the apply-ghc-options setting

schell · 2016-07-20T17:07:07Z

It seems I'm also running into this.

stack --version
Version 1.1.2 x86_64 hpack-0.14.0

My build was failing like so

    /Users/schell/Code/shapes/shapes-math/src/Shapes/Linear/Double.hs:1:1:
        cannot find normal object file ‘.stack-work/dist/x86_64-osx/Cabal-1.22.5.0/build/math-bench/math-bench-tmp/Shapes/Linear/Template.dyn_o’
        while linking an interpreted expression

The executable that was failing in this case is math-bench. The file seems to have some template haskell in it - I'm not sure if that is the issue, but math-bench also has some ghc-options in its cabal file that may be (part of) the culprit

  ghc-options:         -O2
                       -threaded
                       -fexcess-precision
                       -optc-O3
                       -optc-ffast-math
                       -funfolding-use-threshold=16

Since I have access to the cabal file I just set buildable: False and stack seems to be able to build now.

jacereda · 2016-09-23T12:41:25Z

I've hit this problem trying to build HaRe with --profile on both Mac and Windows.

jchia · 2017-02-10T08:12:19Z

Could this be related? haskell/cabal#2827

chreekat · 2017-02-22T01:07:47Z

I also just hit this while building stack HEAD (078cfad) with --executable-profiling. Kind of annoying, since I wasn't getting it just earlier today. :)

I don't have useful report info yet, I'm just pinging the thread. I'm going to blow away .stack-work and see what changes.

chreekat · 2017-02-22T01:51:03Z

My repro steps were:

Install from get.haskellstack.org.

$ stack --version
Version 1.3.3, Git revision f5180d48084091cb975da2bfa70f4aaa51d26abb (dirty) (4595 commits) x86_64 hpack-0.17.0

Go into stack repo and build with profiling.

$ rm -fr .stack-work
$ stack build --executable-profiling
<snip>
    Preprocessing executable 'stack' for stack-1.3.3...
    [1 of 2] Compiling Paths_stack      ( .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/autogen/Paths_stack.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Paths_stack.p_o )
    [2 of 2] Compiling Main             ( src/main/Main.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Main.p_o )
    /home/b/src/Haskell/stack/stack/src/main/Main.hs:127:10:
        cannot find normal object file ‘.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Paths_stack.dyn_o’
        while linking an interpreted expression
    Completed 26 action(s).

    --  While building package stack-1.3.3 using:
          /home/b/src/Haskell/stack/stack/.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/setup/setup --builddir=.stack-work/dist/x86_64-linux/Cabal-1.22.7.0 build lib:stack exe:stack --ghc-options " -ddump-hi -ddump-to-file"
        Process exited with code: ExitFailure 1

Look around for Paths_stack.

$ find .stack-work -name '*Paths_stack*'
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Paths_stack.p_o
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Paths_stack.p_hi
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/autogen/Paths_stack.dump-hi
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/Paths_stack.p_o
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/Paths_stack.p_hi
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/autogen/Paths_stack.hs
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/autogen/Paths_stack.dump-hi
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/Paths_stack.o
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/Paths_stack.hi
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/Paths_stack.dyn_hi
.stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/Paths_stack.dyn_o

So there's one in there, just not where stack wants to find it.

Next is the fun part... overcoming the issue.

Rebuild/install without profiling. No cleaning or anything.

$ stack install
...
$ stack --version
Version 1.3.3, Git revision 078cfadeb37a39501eae24732e5c757cc8aca31b (4608 commits) x86_64 hpack-0.17.0

Build with profiling, again.

stack-1.3.3: unregistering
stack-1.3.3: configure (lib + exe)
Configuring stack-1.3.3...
stack-1.3.3: build (lib + exe)
Preprocessing library stack-1.3.3...
Preprocessing executable 'stack' for stack-1.3.3...
[2 of 2] Compiling Main             ( src/main/Main.hs, .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack-tmp/Main.p_o )
Linking .stack-work/dist/x86_64-linux/Cabal-1.22.7.0/build/stack/stack ...
stack-1.3.3: copy/register
Installing library in
/home/b/src/Haskell/stack/stack/.stack-work/install/x86_64-linux/lts-6.25/7.10.3/lib/x86_64-linux-ghc-7.10.3/stack-1.3.3-Il62v6TwN0R40oGLVzWaiu
Installing executable(s) in
/home/b/src/Haskell/stack/stack/.stack-work/install/x86_64-linux/lts-6.25/7.10.3/bin
Registering stack-1.3.3...

Success!

mgsloan · 2017-02-24T00:07:10Z

@chreekat Could this be the same issue described in #2903 ? Namely, there's appears to be poor interaction between --profile and TH

chreekat · 2017-02-24T00:12:04Z

@mgsloan I would say most likely. Interesting in this case that it's the Paths_<lib>.hs module that causes the trouble, which is autogenerated (but I forget who does the autogeneration).

mgsloan · 2017-02-27T20:29:29Z

Interesting in this case that it's the Paths_.hs module that causes the trouble, which is autogenerated (but I forget who does the autogeneration).

The autogeneration of Paths_ is part of Cabal. No TH involved. It is rather curious that it's failing due to that.

chrissound · 2018-04-09T09:32:56Z

build:
  library-profiling: true
  executable-profiling: true

I was getting similar error messages with the above in my stack.yaml - had to set the values to false to successfully build. Not too sure how relevant this is..

snoyberg · 2019-03-25T08:18:03Z

This appears to be an upstream issue, either in Cabal or GHC. Closing in favor of haskell/cabal#5961

snoyberg · 2019-03-25T08:42:26Z

And I remembered the resolution for this problem: can you try adding other-extensions: TemplateHaskell to the appropriate stanza and see if that fixes things?

mgsloan added the type: bug label Jan 21, 2016

mgsloan added this to the P2: Should milestone Jan 21, 2016

mgsloan added the needs reproduction label Apr 25, 2016

sjakobi mentioned this issue Aug 9, 2016

Linker error when compiling with --trace/--profile #2422

Closed

snoyberg closed this as completed Mar 25, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

--profile and -dynamic builds fail #1691

--profile and -dynamic builds fail #1691

blitzcode commented Jan 21, 2016

sjakobi commented Apr 25, 2016 •

edited

Loading

mgsloan commented Apr 25, 2016 •

edited

Loading

sjakobi commented Apr 26, 2016

sjakobi commented Apr 26, 2016

sjakobi commented Apr 27, 2016

mgsloan commented Apr 27, 2016

schell commented Jul 20, 2016

jacereda commented Sep 23, 2016

jchia commented Feb 10, 2017

chreekat commented Feb 22, 2017

chreekat commented Feb 22, 2017

mgsloan commented Feb 24, 2017

chreekat commented Feb 24, 2017

mgsloan commented Feb 27, 2017

chrissound commented Apr 9, 2018

snoyberg commented Mar 25, 2019

snoyberg commented Mar 25, 2019

--profile and -dynamic builds fail #1691

--profile and -dynamic builds fail #1691

Comments

blitzcode commented Jan 21, 2016

sjakobi commented Apr 25, 2016 • edited Loading

mgsloan commented Apr 25, 2016 • edited Loading

sjakobi commented Apr 26, 2016

sjakobi commented Apr 26, 2016

sjakobi commented Apr 27, 2016

mgsloan commented Apr 27, 2016

schell commented Jul 20, 2016

jacereda commented Sep 23, 2016

jchia commented Feb 10, 2017

chreekat commented Feb 22, 2017

chreekat commented Feb 22, 2017

mgsloan commented Feb 24, 2017

chreekat commented Feb 24, 2017

mgsloan commented Feb 27, 2017

chrissound commented Apr 9, 2018

snoyberg commented Mar 25, 2019

snoyberg commented Mar 25, 2019

sjakobi commented Apr 25, 2016 •

edited

Loading

mgsloan commented Apr 25, 2016 •

edited

Loading