[nix-local-build] Replacement needed for cabal copy

[acfoltzer]

When preparing binary distributions of Cabalized software, I use a combination of configure and copy to prepare the staging directory that will be turned into a tarball, .msi, etc. For example:

% cabal configure --prefix=/usr/local
% cabal build
% cabal copy --destdir=$(pwd)/foo-1.0
% tar -czf foo-1.0.tar.gz foo-1.0 # for example

It looks like we can still specify a --prefix argument, but there is no new version of copy. I imagine a solution for this might overlap with #3332 but I want to make a separate ticket since the use cases are distinct.

[dcoutts]

Yeah, honestly I'm not sure yet what the solution for this looks like. If the goal is to produce an installable exe then there's two cases I think

• you're only producing a statically linked exe and you're lucky enough to not depend on any libraries that use data files
• all other cases, producing dynamically linked exe or depend on libs that have data files

In the former case you can get away with just copying the output of this one package, and build it against deps from the store, but in the latter case you really need to also copy things from those deps. So in the latter case if we're not yet able to do relocatable builds then in fact we need to build the deps into some local space where we've configured all the deps for the user's chosen --prefix and then finally copy all of them to the target image dir.

So to be clear, I think the replacement here is not for copy which had a very limited (single-package) scope, but a command to produce an installable image of the target (and if necessary its deps).

[acfoltzer]

So to be clear, I think the replacement here is not for copy which had a very limited (single-package) scope, but a command to produce an installable image of the target (and if necessary its deps).

Agreed. We structure our builds carefully to avoid problems with data files in dependencies, but this is shaky at best. It would be much nicer to have a more robust solution.

Does it make sense to try the "build deps with --prefix for staging" approach in the short term? Unless relocatable builds are coming soon, I don't want this to be a blocker for us adopting the new commands.

[ezyang]

Hello @acfoltzer,

Here's a workaround to get copy: run cabal copy --destdir=$(pwd)/foo-1.0 --builddir=dist-newstyle/build/foo-1.0/. The directory formats are compatible so this should just work.

Can you see if this does work? And if so, does this move this off the hotpath?

[acfoltzer]

This doesn't seem to work entirely, though I might be Doing It Wrong.

The root problem seems to be that --prefix just isn't handled properly. This is probably worth splitting out into a separate ticket, but I end up with a destdir with the relevant parts of my ~/.cabal grafted in:

cryptol-2.4.0-Darwin-x86_64_2e15d4f
├── Users
│   └── acfoltzer
│       └── .cabal
│           ├── bin
│           │   └── cryptol
│           ├── lib
│           │   └── x86_64-osx-ghc-7.10.3
│           │       └── .fake.cryptol-2.4.0
│           │           ├── Cryptol
│           │           │   ├── Eval
│           │           │   │   ├── Arch.dyn_hi
│           │           │   │   ├── Arch.hi
│           │           │   │   ├── Env.dyn_hi
│           │           │   │   ├── Env.hi
...

[acfoltzer]

Ah, it looks like that copy was picking up the --prefix of a previous cabal configure. It looks like new-configure and new-build don't handle --prefix at all; again an issue for a separate ticket.

[dcoutts]

So our latest thinking on this issue is that we can configure with a custom --prefix etc and still install into the store. Then the replacement for copy (perhaps "image", as in create an installable image) would just copy the relevant files from deps from the store. For static linking that'd just be data files, but for dynamic linking it'd also be shared libs. That works for exes. Doing copy/image for a lib rather than an exe is still "interesting" but should be doable. Fortunately it's not that common to want to build a lib here and deploy it over there so we can postpone that for a bit (though some Haskell progs do rely on building code and so do use libs, e.g. xmonad).

One adjustment that would be necessary for packages with custom prefix in the store is to adjust the paths that get registered with in the ghc-pkg db so that the registration actually works.

[ezyang]

@acfoltzer I assigned you to this ticket because it's in Galois' bucket list. When Galois does get an engineer working on this, reassign it to them!

[acfoltzer]

@ezyang cool; feel free to do this on others where appropriate!