Adding --static flag for building static executables with stack?

[Tehnix]

A great question was posed in the comments to this article https://www.fpcomplete.com/blog/2016/10/static-compilation-with-stack, although it seems to have been left unanswered.

Mainly, if it could be possible to have a --static flag in stack, that would set the flags necessary for building a statically linked binary, and perhaps be aware of the crtbeginT and suggest/link to the proposed fix.

This issue #1032 which is still open, seems to be somewhat related, but has been left without any activity (except for a few semi-related comments) for quite a while.

Any thoughts if it is even feasible to begin such work?

[erebe]

Hello,
The crtbeginT hack is not at all required to compile a static binary.
The only thing needed (beside static versions of librairies) is to add the option ld-options: -static in your cabal file and compile with --ghc-options="-fPIC"

Please see this
https://stackoverflow.com/questions/41419102/haskell-stack-static-binary-relocation-r-x86-64-32-against-tmc-end-can-not/41427067#41427067 and https://www.reddit.com/r/haskell/comments/5lk33p/struggling_building_a_static_binary_for_aws/
for an in-depth why.

I would like to fight this miss-conception of the crtbeginT hack, as it is mentioned almost everywhere when this subject is talked about and too many post now refer to it.
For me the easiest option to do that would be to add an --static option to stack.

I tried to grok the code in order to propose a PR, but sadly I don't understand zip of the code.
If someone can guide me trough it so I can add the option.

[mgsloan]

Would certainly be nice to support this without modification of the cabal package, though.

So maybe by having --static. Alternatively, though more manual, should have --configure-options (#1438) now that some of the details of ghc-options + flags have been sussed out.

[Tehnix]

@erebe ah, nice! I found several sources that mentioned the crtbeginT hack, am glad to see it's not necessary!

@mgsloan Am I correct in the understanding that --configure-options would be able to pass the ld-options: -static if we wanted? Cause then a --static seems to be simply an alias for stack install --ghc-options="-fPIC" --configure-options="ld-options: -static" (or something like that). Would greatly simplify things!

[mgsloan]

@Tehnix Yep, that's right!

[erebe]

So how can we go further with this request ?
If we follow those tickets, every single one is marked as closed or blocked.

Do you see a way @mgsloan to unlock the situation in order to get a --static ?
If you provide me some support/direction, I can spend some time to propose something.

[mgsloan]

I'd say it's reasonable to unblock that issue.

Unfortunately, it's probably trickier than I'd thought, if you need everything to be built with -fPIC. There's no straightforward way to reliably do this. The problem is that snapshot packages do not get rebuilt, even if they were built with different options. Workarounds:

• Blow away your stack root whenever you want to recompile snapshot packages statically.

• Use stack list-dependencies on your snapshot-using project, and put the results in extra-deps (removing some special cases like ghc and other wired in packages).

• Wait for implicit snapshots is implemented -
  Implicit snapshots #3330 . Not sure when that will happen, it is a big undertaking.

• Implement --static as described below, delete your STACK_ROOT, and put static-executables: true in your global configuration.

Even though it wouldn't work ideally, it makes sense to me to implement --static, and have its documentation indicate that it is experimental and requires special care for it to work right (the stuff options listed above)

Here's what the implementation would look like:

• Add it to Config and ConfigMonoid in Stack.Types.Config.
• Add it to cli options parsing for ConfigMonoid, and static-executables: true option to the config (is that a good name for the config option? static-build: true since it affects more than just exes?)
• Check if the flag is set in Stack.Types.Build.configureOptsNoDirs, if it is, pass the appropriate configure flags.

I'd accept a PR for this, even if the more complicated issue of snapshot package options isn't resolved.

[erebe]

ack, I am finishing some work on an other project and will try to submit something after that

[erebe]

I have opened an issue in cabal haskell/cabal#4925, as after a few hours I didn't understood why my flag options were not taken into account

[nh2]

I have PR'd a feature to Cabal that will be necessary to make this work, or at least make it easier:

--enable-executable-static haskell/cabal#5446

[nh2]

Beyond that, we need to make it easy to provide a libc that can be easily linked statically (like musl), or figure out how to convince glibc to be linked statically (but beware, glibc is LGPL licensed, you are not allowed to statically link it in proprietary projects).

• glibc static linking FAQ
• https://github.com/nh2/static-haskell-nix has the musl route working in nix
  • I am thinking we may use nix to provide a musl-supporting GHC and required non-Haskell libraries to provide stack with a compiler and system environment so that it can use --enable-executable-static itself

[dbaynard]

@nh2 is #4088 related here? As in, if stack can build static binaries, it can build itself as a static binary.

[nh2]

@dbaynard Yes, slightly related, exactly as you said: Solving this one automatically makes #4088 trivial.

However #4088 is simpler than this one, as it can also be solved by building stack statically via Alpine or Nix.

[ryukinix]

How this task is working on? I'm interested with it, seems that the crtbeginT hack it still necessary for me, but even with it I didn't successfully get. Maybe I'm doing something outdated/wrong?

Anyone can share workarounds to achieve this without the hack? Currently I'm doing this:

stack  build --ghc-options='-optl-static -optl-pthread -fPIC -L$(MUSL_PATH)'    --force-dirty	

Output:

clap-counter-0.0.0: build (lib + exe)                                                                                                                
Preprocessing library for clap-counter-0.0.0..                                                                                                       
Building library for clap-counter-0.0.0..                                                                                                            
[6 of 6] Compiling Application      ( src/Application.hs, .stack-work/dist/x86_64-linux-tinfo6/Cabal-2.4.0.1/build/Application.o ) [flags changed]   
/usr/bin/ld.gold: error: /usr/lib/gcc/x86_64-pc-linux-gnu/8.2.1/crtbeginT.o: requires dynamic R_X86_64_32 reloc against '__TMC_END__' which may overf
low at runtime; recompile with -fPIC                                                                                                                 
collect2: error: ld returned 1 exit status                                                                                                           
`gcc' failed in phase `Linker'. (Exit code: 1)                                                                                                       
                                                                                                                                                     
--  While building package clap-counter-0.0.0 using:                                                                                                 
      /home/lerax/.stack/setup-exe-cache/x86_64-linux-tinfo6/Cabal-simple_mPHDZzAJ_2.4.0.1_ghc-8.6.3 --builddir=.stack-work/dist/x86_64-linux-tinfo6/
Cabal-2.4.0.1 build lib:clap-counter exe:clap-counter --ghc-options " -ddump-hi -ddump-to-file -fdiagnostics-color=always"                           
    Process exited with code: ExitFailure 1                                                                                                          

Where MUSL_PATH it's a path containing the .a static libraries of libc. I'm pretty newbie with stack, mostly used cabal for all life. Sorry if I'm doing something completly wrong (like asking this here (?)).

[ryukinix]

Just to share for someone lost as me: https://www.reddit.com/r/haskell/comments/9on8gi/is_there_an_easy_way_to_compile_static_binaries/

This helped me a lot!

Sorry for the annoying comments. Bye

[nh2]

@ryukinix For posterity, which change fixed your exact problem above?

[ketzacoatl]

@nh2 In my case, I followed the first reasonable suggestion from that thread, updating the ghc-options and adding cc-options and ld-options to package.yaml with:

executables:
  ops:
    ghc-options:
    - -threaded
    - -static
    - -O2
    cc-options: -static
    ld-options: -static -pthread

..and then running stack build, successfully re-built my project with a statically-linked executable:

file ops
ops: ELF 64-bit LSB executable, x86-64, version 1 (SYSV), statically linked, for GNU/Linux 3.2.0,

It was easy, and I was surprised.

Basic tests of the executable have passed (the tool works as expected). I was able to use this to build the executable on ubuntu 18.04 that runs fine on ubuntu 16.04.

[krakrjak]

The suggested solution doesn't seem to work any longer on Ubuntu. I'm on 20.04 (focal) and I get errors as documented in icfpcontest2020/starterkit-haskell#2

[avanov]

@nh2 regarding #3420 (comment) , LGPL does allow static linking against proprietary software under a condition that the proprietary software, whilst being distributed as a fully compiled program, also provides itself in either source code (not suitable for most commercial projects) or compiled object code that can be linked together with the LGPL code in question (suitable for commercial projects, but requires extra work).

https://www.gnu.org/licenses/gpl-faq.en.html#LGPLStaticVsDynamic