static build with openssl causes segv

[maiha]

Program crashes when using static build with openssl such as http.

% cat app.cr
require "http"
p 1

% crystal build app.cr --release --link-flags "-static"
% ./app
[1]    27887 segmentation fault (core dumped)  ./app

Full backtraces of the core dump is here.
https://gist.github.com/maiha/b008e524558fe1e5ecc9e34568dd8074

And, using http without openssl works well.

% crystal build app.cr --release --link-flags "-static" -D without_openssl
% ./app
1

Envs.

% crystal -v
Crystal 0.18.7 [68783f1] (2016-07-03)

% uname -a
Linux ubuntu 4.4.0-31-generic #50-Ubuntu SMP Wed Jul 13 00:07:12 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

thanks.

[lribeiro]

I was about to create a new issue with this, but I checked first and found this was already open.
Still happens on 0.19.1, any ideas how one would try to fix it?

uname -a
Linux fd3f38b13502 4.4.19-moby #1 SMP Thu Sep 1 09:44:30 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

crystal --version
Crystal 0.19.1 [798b2e2] (2016-09-09)

# a.cr
require "openssl/cipher"
p 1

Build the static version

crystal build --release --link-flags "-static -L/opt/crystal/embedded/lib" a.cr 
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libcrypto.a(fips.o): In function `verify_checksums':
(.text+0x4e6): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/crystal/embedded/lib/libevent.a(evutil.o): In function `test_for_getaddrinfo_hacks':
evutil.c:(.text+0x1488): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/crystal/embedded/lib/libevent.a(evutil.o): In function `evutil_unparse_protoname':
evutil.c:(.text+0xf0d): warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

Running the static version

./a
Segmentation fault (core dumped)

GDB backtraces
https://gist.github.com/lribeiro/d0d866fa9ce2abe326fb598b6a844bc7

[lribeiro]

Pin pointed to LibCrypto

require "openssl/lib_crypto"
LibCrypto.openssl_add_all_algorithms
puts 1

[ysbaddaden]

Thanks for reducing. It eventually tries to access 0x0 (null pointer?), which seems to be the value of symbolp@entry. I have no idea why.

#1 0x00007fc08cd3aa33 in __GI__dl_addr (address=address@entry=0x7fc08cc7cde0 <ptmalloc_init>, info=info@entry=0x7ffd2b6bc060, mapp=mapp@entry=0x7ffd2b6bc058, symbolp=symbolp@entry=0x0) at dl-addr.c:131

Also, LibCrypto is statically linked, but it still uses dlopen?

[RX14]

Seems statically linking with libcrypto at all needs special work. The linker is the same for C and crystal applications so this answer may provide the solution.

I would say with 99% certainty this isn't a crystal bug, but openssl being weird.

[lribeiro]

@RX14 thanks, the solution from that stackoverflow doesn't work for me. But it did gave some clues on how to proceed.

@ysbaddaden this might explain dlopen even with static

Compiled openssl master branch from source with no-shared to make it static and for some reason it doesn't have OPENSSL_add_all_algorithms_noconf exported on libcrypto.a.

#Openssl compiled from source master branch
crystal build a.cr  --link-flags "-L/home/openssl -static"

/opt/crystal/embedded/lib/../lib/libevent.a(evutil.o): In function `test_for_getaddrinfo_hacks':
evutil.c:(.text+0x1488): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/crystal/embedded/lib/../lib/libevent.a(evutil.o): In function `evutil_unparse_protoname':
evutil.c:(.text+0xf0d): warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
_main.o: In function `__crystal_main':
main_module:(.text+0x44c): undefined reference to `OPENSSL_add_all_algorithms_noconf'
collect2: error: ld returned 1 exit status

Checked the docker container version and it had 1.0.2g (Ubuntu 16.04)
Compiled branch 1.0.2 stable from with the following options

#compiling openssl with static libcrypto.a
git clone https://github.com/openssl/openssl.git
cd openssl
./config no-shared
make depend
make -j9

# Testing for OPENSSL_add_all_algorithms_noconf exported
$> grep OPENSSL_add_all_algorithms_noconf libcrypto.a
Binary file libcrypto.a matches

Compiled with the openssl static version

#a.cr
require "openssl/lib_crypto"
LibCrypto.openssl_add_all_algorithms
puts 1

$>  crystal build a.cr  --link-flags "-ldl -L/home/openssl -static"
/home/openssl/libcrypto.a(dso_dlfcn.o): In function `dlfcn_globallookup':
dso_dlfcn.c:(.text+0x11): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/crystal/embedded/lib/../lib/libevent.a(evutil.o): In function `test_for_getaddrinfo_hacks':
evutil.c:(.text+0x1488): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/opt/crystal/embedded/lib/../lib/libevent.a(evutil.o): In function `evutil_unparse_protoname':
evutil.c:(.text+0xf0d): warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

Now it works

$> ./a
1

#Checking the file is indeed statically linked
$> file a
a: ELF 64-bit LSB executable, x86-64, version 1 (GNU/Linux), statically linked, for GNU/Linux 2.6.32, BuildID[sha1]=e867c64a8449f0474b792b402783e9106c26c9d5, not stripped

[rdp]

Seems to link OK static for me now: 0.31.1 linux :)

[straight-shoota]

It links, but still causing linker warnings (using docker image crystallang/crystal:0.31.1):

$ cat > test.cr
require "openssl/lib_crypto"
LibCrypto.openssl_add_all_algorithms
puts 1
$ crystal build --static test.cr
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libcrypto.a(dso_dlfcn.o): In function `dlfcn_globallookup':
(.text+0x11): warning: Using 'dlopen' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libevent.a(evutil.o): In function `test_for_getaddrinfo_hacks':
(.text+0x144): warning: Using 'getaddrinfo' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking
/usr/lib/gcc/x86_64-linux-gnu/5/../../../x86_64-linux-gnu/libevent.a(evutil.o): In function `evutil_getaddrinfo_common':
(.text+0x1749): warning: Using 'getprotobynumber' in statically linked applications requires at runtime the shared libraries from the glibc version used for linking

[straight-shoota]

On alpine it links without warnings. Had to use a different function because LibCrypto.openssl_add_all_algorithms is only available for older versions of OpenSSL. I switchted to evp_md4, which causes the same linker errors on Ubuntu as openssl_add_all_algorithms.

Since static linking only really works on musl anyway, I think it's fine.