Missing shared library in Linux Binary

[chriskolenko]

Tried to run rattletrap inside a docker container, but I'm missing some shared libraries.

Can you statically link them when compiling the binary? Not a haskell dev sorry

[chriskolenko]

Installing libgmp-dev got the binary working. Is there anyway to remove this dependency?

[tfausak]

Sorry about that! The Haskell runtime requires libgmp. I should be able to statically link it when building the binaries.

[chriskolenko]

I also needed to install build-essentials. Don't know why exactly, something to do with http when passing in a URI as the Input.

[tfausak]

HTTPS requests require some combination of ca-certificates and netbase. I’m not sure what regular HTTP requires.

…

Sent from my iPhone

On Mar 25, 2018, at 10:16 AM, Chris ***@***.***> wrote: I also needed to install build-essentials. Don't know why exactly, something to do with http when passing in a URI as the Input. — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

[tfausak]

As far as I know there's no way to bundle the stuff needed for HTTPS, but statically linking libgmp shouldn't be too hard.

https://ro-che.info/articles/2015-10-26-static-linking-ghc
https://www.fpcomplete.com/blog/2016/10/static-compilation-with-stack
commercialhaskell/stack#3420

[tfausak]

Alright, I've finally got some time to work on this. I started a new branch in order to test some static compilation on Travis CI. Building the executable the old way (that is, dynamically linked) gave this output:

$ ls -h -l $( stack exec which rattletrap )
-rwxr-xr-x 1 travis travis 23M Jul  7 14:57 /home/travis/build/tfausak/rattletrap/.stack-work/install/x86_64-linux/lts-11.0/8.2.2/bin/rattletrap

$ ldd $( stack exec which rattletrap )
	linux-vdso.so.1 =>  (0x00007ffe8e9e4000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fcf9043e000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fcf90220000)
	libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fcf90007000)
	librt.so.1 => /lib/x86_64-linux-gnu/librt.so.1 (0x00007fcf8fdff000)
	libutil.so.1 => /lib/x86_64-linux-gnu/libutil.so.1 (0x00007fcf8fbfc000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fcf8f9f8000)
	libgmp.so.10 => /usr/lib/x86_64-linux-gnu/libgmp.so.10 (0x00007fcf8f784000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fcf8f56e000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fcf8f1a5000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fcf90744000)

Attempting a static build (predictably) failed:

/usr/bin/ld.gold: error: /usr/lib/gcc/x86_64-linux-gnu/4.8/crtbeginT.o: requires dynamic R_X86_64_32 reloc against '__TMC_END__' which may overflow at runtime; recompile with -fPIC
collect2: error: ld returned 1 exit status
`gcc' failed in phase `Linker'. (Exit code: 1)
--  While building custom Setup.hs for package rattletrap-4.1.1 using:
      /home/travis/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.0.1.0 build lib:rattletrap exe:rattletrap test:test --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1

I'm running another build with the -fPIC option, but I suspect that will fail too. I think the right solution is using ld-options: -static. I'll try that after the PIC flag build completes.

[tfausak]

Yup, similar problem with -fPIC:

/usr/bin/ld.gold: error: /usr/lib/gcc/x86_64-linux-gnu/4.8/crtbeginT.o: requires dynamic R_X86_64_32 reloc against '__TMC_END__' which may overflow at runtime; recompile with -fPIC
collect2: error: ld returned 1 exit status
`gcc' failed in phase `Linker'. (Exit code: 1)
--  While building custom Setup.hs for package rattletrap-4.1.1 using:
      /home/travis/.stack/setup-exe-cache/x86_64-linux/Cabal-simple_mPHDZzAJ_2.0.1.0_ghc-8.2.2 --builddir=.stack-work/dist/x86_64-linux/Cabal-2.0.1.0 build lib:rattletrap exe:rattletrap test:test --ghc-options " -ddump-hi -ddump-to-file"
    Process exited with code: ExitFailure 1

Fortunately the ld-options approach is looking a lot better:

$ ls -h -l $( stack exec which rattletrap )
-rwxr-xr-x 1 travis travis 25M Jul  7 15:16 /home/travis/build/tfausak/rattletrap/.stack-work/install/x86_64-linux/lts-11.0/8.2.2/bin/rattletrap

$ ldd $( stack exec which rattletrap )
	not a dynamic executable

However there are a bunch of linking warnings like this:

/home/travis/.stack/snapshots/x86_64-linux/lts-11.0/8.2.2/lib/x86_64-linux-ghc-8.2.2/network-2.6.3.4-D6yy3exF2RyArU0wk1GgIu/libHSnetwork-2.6.3.4-D6yy3exF2RyArU0wk1GgIu.a(BSD.o)(.text+0xab9f): warning: Using 'endservent' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

A quick look around makes me think that those linking warnings will be difficult to work around. Doing this properly might require building with musl libc, which isn't something I know how to do on Travis CI.

[tfausak]

The binaries on the releases page are still dynamically linked, but there is now a Docker image with a statically linked binary: https://hub.docker.com/r/taylorfausak/rattletrap/

$ docker run --rm taylorfausak/rattletrap rattletrap --input https://github.com/tfausak/rattletrap/raw/master/replays/29f5.replay | jq .header.body.properties.value.Date
{
  "kind": "StrProperty",
  "size": 21,
  "value": {
    "str": "2016-06-06:22-12"
  }
}