Allow v2-install to copy executables instead of symlinking

[matil019]

Nice to meet you!

I'd like to request a feature, which allows users to install executables with cabal v2-install as an actual copy of the executables instead of symlinks to them.

The reason is that if symlinks are used, removing ~/.cabal/store is no longer safe, in contrary to be described in the user guide:

This split motivates some of the UI choices for Nix-style local build commands. For example, flags passed to cabal new-build are only applied to local packages, so that adding a flag to cabal new-build doesn’t necessitate a rebuild of every transitive dependency in the global package store.

[fgaz]

removing ~/.cabal/store is no longer safe

Note that it may not be safe even if exes are copied, for example in the presence of data files

[fgaz]

Also what is your goal? Packaging/distribution, just freeing up space (#3333) or something else entirely?

[matil019]

I often write small programs in Haskell for personal use, for experiments, and to understand Haskell better. I mess up cabal builds quite often, so I'd like a "reset button". I've been accustomed to removing ~/.ghc but the v2- commands don't appear to be using it. I hesitate to remove whole ~/.cabal because 1) I don't want to download package db and tarballs, and 2) I don't want to remember what hand-made programs of mine were built and installed with cabal.

Freeing up space is not a primary concern but I'd like to keep a safe way to clean up cache at hand. Rebuilding libraries in Hackage is ok but breaking (my) executables is not. I don't intend to distribute them (yet).

[fgaz]

reset button

Not needed anymore with v2-* :)

Packages are now isolated and cannot interfere with each other anymore, so the only reason for deleting the store should be lack of disk space (well, or a corrupted store in case of a power failure in the middle of a build. see #5529)

[matil019]

Not needed anymore with v2-* :)

Nice to hear that.

As an alternative to waiting for #3333, is there a way to un-cabalize a package so that the executable is independent of ~/.cabal/store? Copying files manually is enough? (I don't use "data files" now, but if there is a way to cover them too, it's better.)

I tried cabal v2-install --global --prefix=foo exe:bar but it appears that v2-install ignores the options. And there isn't v2-copy.

[hvr]

@matil019 maybe the discussion/suggestions at #5551 (comment) could be of interest to you

[fommil]

My usecase is that when I build binaries inside a docker container, the symlinks are no longer visible to the outer container. https://gitlab.com/fommil/dotfiles/blob/master/bin/cabal-docker

@hvr doh, I've minimised my original comment to avoid the distraction.

[fgaz]

Implemented in #5870. Note that #5942 is still unfixed

[jberryman]

This isn't really fixed right? Since "removing ~/.cabal/store is no longer safe, in contrary [sic] to be described in the user guide" (since some executables may depend on data files, as documented).

Losing all the tooling one depends on by removing a cache directory is a big deal. Hopefully most binaries installed with the new (old) copy functionality will function after clobbering the store, but I really have no idea.

[hvr]

@jberryman well, yes, this was pointed out right away in the first comment at #5628 (comment) that the best we can do at this point is to add support for copying instead of symlinking the executable (only).

It's not clear how we can achieve the ideal goal of being able to "copy" executables out of the Nix-style store in a self-contained way no longer retaining any reference to the store in a portable way. The way the cabal spec works, it is currently IMO not possible to end up with completely self-contained executables when data-files/libexec/etc are involved.

The next best thing we could do however is to collect all data-files and libexec programs and whatnot of an executable and its transitive dependencies, and place them into a "bundle folder" and generate wrapper scripts which set the respective environment variables which would allow the executable to locate their auxilary data that would have lived in the Nix-style store. But it won't be a self-contained executable.

But we should also take a step back and ensure this isn't an XY problem; is it just about resetting the Nix-style ~/.cabal/store from time to time, or is it like about generating binary distributions? For the former case the better approach is to implement store GC (see #3333), and for the binary distribution case I'm not sure you want to work from Nix-style store cached artifacts and instead build binaries appositely with the proper locations for data-files (living in the respective filesystem's conventional places, be it /opt/myapp/{lib,libexec,data,...}or/usr/...or/usr/local/...` or whatever) compiled into to avoid requiring env-var-setting wrapper shell scripts.

[jberryman]

we should also take a step back and ensure this isn't an XY problem

Although they proposed copying binaries I think @matil019 is quite clear about their concern (which is also my own): if ~/.cabal/store needs to be deleted for whatever reason (I did it most recently after hacking on GHC; next time I'll try to remember to specify a different store (every time)) this is a bad (possibly unrecoverable) situation for those relying on cabal install-ed binaries. And if cabal (new-)install is not suitable for binary distribution it's not clear to me why it exists at all.

I just don't want that point to get lost. I can certainly appreciate this is a tricky (and pre-existing) problem.

[gbaz]

Could we add a flag when v2-installing a binary to specify a data dir (or a prefix to datadirs) that's not in the store? And then have the same confirm/overwrite logic we have for executable copying?