Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build error of hackage2nix executable in windows with openssl-1.0.2j-fips-x86_64 #389

Closed
jneira opened this issue Nov 22, 2018 · 14 comments
Closed

Comments

@jneira
Copy link

jneira commented Nov 22, 2018

Hi, I've tried to build the lib in windows:

extra-include-dirs: X:\path\to\OpenSSL-x86_64\include
extra-lib-dirs: X:\path\to\OpenSSL-x86_64\lib
  • executing cabal new-build -v3 in mingw (to allow it build network package)

It fails building the executable stack2nix hackage2nix

[3 of 3] Compiling Main             ( hackage2nix\Main.hs, D:\ws\eta\dhall\cabal
2nix-2.11.1\dist-newstyle\build\x86_64-windows\ghc-8.4.4\cabal2nix-2.11.1\x\hack
age2nix\build\hackage2nix\hackage2nix-tmp\Main.o )
Linking D:\ws\eta\dhall\cabal2nix-2.11.1\dist-newstyle\build\x86_64-windows\ghc-
8.4.4\cabal2nix-2.11.1\x\hackage2nix\build\hackage2nix\hackage2nix.exe ...
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x59): referencia a `inflate' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0xea): referencia a `deflate' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x13d): referencia a `inflateEnd' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x158): referencia a `deflateEnd' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x1c3): referencia a `inflateEnd' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x1cc): referencia a `deflateEnd' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x268): referencia a `inflateInit_' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x2d1): referencia a `deflateInit_' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x526): referencia a `deflate' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x5cf): referencia a `zError' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x7cc): referencia a `deflate' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x84f): referencia a `zError' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x8de): referencia a `deflateInit_' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0x9a3): referencia a `inflate' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0xa11): referencia a `zError' sin definir
X:/path/to/OpenSSL-x86_64/lib/libcrypto.a(c_zlib.o):c_zlib.c:(.
text+0xa83): referencia a `inflateInit_' sin definir
collect2.exe: error: ld returned 1 exit status
`gcc.exe' failed in phase `Linker'. (Exit code: 1)
cabal: Failed to build exe:hackage2nix from cabal2nix-2.11.1.
  • cabal version
cabal-install version 2.4.0.0
compiled using version 2.4.0.1 of the Cabal library
  • windows version: 7
  • Build profile: -w ghc-8.4.4 -O1

Thanks in advance!

@peti peti changed the title Build error of stack2nix executable in windows with openssl-1.0.2j-fips-x86_64 Build error of hackage2nix executable in windows with openssl-1.0.2j-fips-x86_64 Nov 23, 2018
@peti
Copy link
Member

peti commented Nov 23, 2018

The source for this error is a broken hopenssl build. Note that hackage2nix never refers to the system's openssl library directly. Instead, it relies on the Haskell package hopenssl to provide the necessary functions for it. Clearly, this isn't working. My guess is that the build needs to link the z library as well, which provides functions like inflate and deflate.

@jneira
Copy link
Author

jneira commented Nov 24, 2018

Oh, i built hopenssl separately and cabal didn't complain about missing z lib, only asked for openssl ones. Should it do it? Or should i tell cabal that it have to call ld including z lib (i dont have much experience building haskell libs that wraps c libs)?

@peti
Copy link
Member

peti commented Nov 24, 2018

i built hopenssl separately and cabal didn't complain about missing z lib, only asked for openssl ones.

The error does not seem to occur until the library is actually used. You might want to try running "cabal test" or "cabal new-test" for hopenssl to see whether those succeed.

@jneira
Copy link
Author

jneira commented Nov 27, 2018

Hi, so a suitable zlib1.dll for the current system is needed in PATH to run executables that includes hopenssl (fe its internal tests). However hopenssl doesnt need it to be built.
After some tests i've found a suitable library for my system in the MINGW64 installation (all standalone downloads of zlib i've found dont work for me). Executing hopenssl tests within MINGW64 doesnt throw a runtime error. Although they all fail like this one:

test\CheckHighLevelDigestAPI.hs:18
SHA256
expected: "41ef4bb0b23661e66301aac36066912dac037827b4ae63a7b1165a5aa93ed4eb"
 but got: "\"41ef4bb0b23661e66301aac36066912dac037827b4ae63a7b1165a5aa93ed4eb\""

As alternative you can use in dos/ps console simply adding the MINGW64 zlib1.dll location in PATH.

Ok, for build cabal2nix we need link statically zlib in hopenssl so the unique way i've managed to do it is:

  • Download hopenssl with cabal get hopenssl-2.2.2
  • Change its cabal file to add z lib as static dependency:
extra-libraries:      crypto
                      z
  • Download cabal2nix with cabal get cabal2nix-2.11.1
  • Make it use my modified local copy of hopenssl with this cabal.project
packages: . ..\hopenssl-2.2.2

I haven't find a cabal option (in cabal.project or .cabal) to make cabal2nix build hopenssl with zlib statically linked so it seems the unique way to make it work would be add z to the hopenssl package extra-lib stanza (like i did).
As zlib is a runtime dependency for hopenssl maybe it should be activated with a dedicated cabal flag -Dlink-zlib or something alike. Or alternatively link it statically if os(windows), if the unique os affected is it. Dont know why it works in linux, maybe my deductions are wrong... 🤔
Maybe no one ever will want to build cabal2nix in windows 😛 , but if we could use nix in mingw maybe it worths the time to do it.
If you think so i could make a pr in hopenssl.
Thanks in advance.

@peti
Copy link
Member

peti commented Nov 27, 2018

Thank you very much for all the effort you've put into debugging this issue! I've opened a new ticket at
peti/hopenssl#4 to remind myself that the hopenssl test suite needs fixing. I'm not sure where that error comes from, but we can probably figure it out.

Adding z to the extra-libraries stanza in hopenssl feels a bit like a kludge. I don't want to add that dependency unconditionally. If I add it conditionally, however, then users who build hopenssl will have to add an appropriate flag on the command line to choose whether to enable that additional dependency or not. Given that situation, I wonder whether maybe it's easier to let users just specify

--ghc-option=-L/path/to/libz --ghc-option=-lz

when they're building hopenssl? I believe that's superior, because ultimately I don't know what kind of run-time dependencies their particular openssl library has. It might depend on libz or not, but it might also depend on additional stuff, i.e. because they have some special crypto hardware that's supported by openssl. So letting everybody specify their additional dependencies manually feels more reliable than trying to guess the properties of their system's binaries.

@jneira
Copy link
Author

jneira commented Nov 27, 2018

Adding z to the extra-libraries stanza in hopenssl feels a bit like a kludge.

Totally agree on that

My first way to do it was trying to configure the build of stack2nix to build in turn hopenssl correctly. I'll try using ghc-option's, thanks!

@peti
Copy link
Member

peti commented Nov 28, 2018

@jneira, I fixed the hopenssl test suite. If you have the chance, please let me know whether building that package with appropriate --ghc-option flags solves your issue!

@jneira
Copy link
Author

jneira commented Nov 29, 2018

Hi, i am been busy testing cabal builds with -v3 and analyzing the results:

  • It seems openssl is built by default without zlib statically linked. At least the version i am using doesn't have it.
  • At first i tried the build with --ghc-option with no luck.
  • So i reviewed the cabal verbose log with extra-libraries: openssl z that works and afaiu cabal passes -lz in a first gcc call and the call to hsc2hs. So i tried --gcc-option=-lz but with no luck 😟

The log for gcc calls are similar, only changes the order of params

  • Version with extra-libraries: z
"D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
"-fno-stack-protector" 
"C:\TEMP\8604-4.c" 
"-o" "C:\TEMP\8604-5" 
"-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen" 
"-ID:\dev\OpenSSL-x86_64\include" 
"-I." 
"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2\include" 
"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0\include" 
"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0\include" 
"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/include" 
"-lcrypto" 
"-lz" 
"-LD:\dev\OpenSSL-x86_64\lib" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\deepseq-1.4.3.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\array-0.5.2.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\ghc-prim-0.5.2.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/rts"
  • Version with --gcc-option=-lz
"D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
"-fno-stack-protector" 
"C:\TEMP\9692-4.c" 
"-o" "C:\TEMP\9692-5" 
"-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen" 
"-ID:\dev\OpenSSL-x86_64\include" 
"-I." 
"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2\include" 
"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0\include" 
"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0\include" 
"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/include" 
"-lcrypto" 
"-LD:\dev\OpenSSL-x86_64\lib" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\deepseq-1.4.3.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\array-0.5.2.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\ghc-prim-0.5.2.0" 
"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/rts" 
"-lz" "-lz"

The log of hsc2hs is similar too:

  • With extra-libraries: openssl z
"D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\bin\hsc2hs.exe" 
"--cc=D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
"--ld=D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
"--cflag=-fno-stack-protector" 
"--lflag=-fno-stack-protector" 
"--cflag=-ID:\dev\OpenSSL-x86_64\include" 
"--cflag=-ID:\dev\OpenSSL-x86_64\include" 
"--cflag=-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen" 
"--cflag=-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\global-autogen" 
"--cflag=-include" 
"--cflag=D:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen\cabal_macros.h" 
"--lflag=-LD:\dev\OpenSSL-x86_64\lib" 
"--lflag=-lcrypto" "--lflag=-lz" 
"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2\include" 
"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0\include" 
"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0\include" 
"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/include" "--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\deepseq-1.4.3.0" "--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\array-0.5.2.0" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0" 
"--lflag=-lwsock32" "--lflag=-luser32" "--lflag=-lshell32" "--lflag=-lmsvcrt" "--lflag=-lmingw32" "--lflag=-lmingwex" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\ghc-prim-0.5.2.0" "--lflag=-luser32" "--lflag=-lmingw32" "--lflag=-lmingwex" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/rts" "--lflag=-lm" "--lflag=-lwsock32" "--lflag=-lgdi32" "--lflag=-lwinmm" 
"--lflag=-lDbghelp" "--lflag=-lPsapi" "-o" "D:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\OpenSSL\EVP\Digest\Algorithm.hs" "src\OpenSSL\EVP\Digest\Algorithm.hsc"
  • Version with --gcc-option=-lz
"D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\bin\hsc2hs.exe" 
"--cc=D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
"--ld=D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
"--cflag=-fno-stack-protector" 
"--cflag=-lz" 
"--cflag=-lz" 
"--lflag=-fno-stack-protector" 
"--lflag=-lz" 
"--lflag=-lz" 
"--cflag=-ID:\dev\OpenSSL-x86_64\include" 
"--cflag=-ID:\dev\OpenSSL-x86_64\include" 
"--cflag=-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen" 
"--cflag=-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\global-autogen" 
"--cflag=-include" 
"--cflag=D:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen\cabal_macros.h" 
"--lflag=-LD:\dev\OpenSSL-x86_64\lib" 
"--lflag=-lcrypto" 
"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2\include" 
"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0\include" 
"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0\include" 
"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/include" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\deepseq-1.4.3.0" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\array-0.5.2.0" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0" 
"--lflag=-lwsock32" "--lflag=-luser32" "--lflag=-lshell32" "--lflag=-lmsvcrt" "--lflag=-lmingw32" "--lflag=-lmingwex" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\ghc-prim-0.5.2.0" "--lflag=-luser32" "--lflag=-lmingw32" "--lflag=-lmingwex" 
"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/rts" "--lflag=-lm" "--lflag=-lwsock32" "--lflag=-lgdi32" "--lflag=-lwinmm" 
"--lflag=-lDbghelp" "--lflag=-lPsapi" "-o" "D:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\OpenSSL\EVP\Digest\Algorithm.hs" "src\OpenSSL\EVP\Digest\Algorithm.hsc"

I've not find any other reference to z lib in the rest of calls of the cabal build with extra-libraries: openssl z. I can share both complete logs if is needed.

@jneira
Copy link
Author

jneira commented Dec 17, 2018

Hi, i've opened an issue in cabal about this: haskell/cabal#5805

peti added a commit to peti/hopenssl that referenced this issue Dec 22, 2018
Static variants of libcrypto may depend on libz but can't express that
dependency, so we have to worry about telling Cabal about it.

Relevant Issue:
 - NixOS/cabal2nix#389
 - haskell/cabal#5805
@peti
Copy link
Member

peti commented Dec 22, 2018

Hey @jneira, the latest git version of hopenssl @ peti/hopenssl@94aa591 has a new Cabal flag link-libz that you can enable to add the missing dependency on libz. It's not pretty, but I figure it should work.

Are you still in the mood for testing the change? Does the new version install correctly for you and allows you to build cabal2nix on Windows? I'll happily make a new release to Hackage once I know it actually helps.

@jneira
Copy link
Author

jneira commented Dec 23, 2018

Yeah, not at home in some days but i will test it asap.
Thanks for your step forward! i hope the cabal issue will give us some better alternative

@jneira
Copy link
Author

jneira commented Dec 26, 2018

Finally i've tried it:

  • hopenssl: cabal new-build --enable-tests -flink-libz works with master version
  • cabal2nix: cabal new-build works in my win 10 migw64 console using this cabal.project:
packages: .

source-repository-package
    type: git
    location: https://github.com/peti/hopenssl.git
    tag: 65fd7f68014b55c9dcc94c6084b52e426e69a802

package hopenssl
    flags: link-libz

Lot of thanks!

@peti
Copy link
Member

peti commented Dec 26, 2018

Very cool. I released hopenssl 2.2.4 to Hackage, which has that flag included. I'm glad we have finally solved this issue. It's not 100% satisfactory, IMHO, but it seems like the best we can do at the moment without resorting to major complications, like using Autoconf, etc. Thank you very much for your bug report.

@peti peti closed this as completed Dec 26, 2018
@jneira
Copy link
Author

jneira commented Sep 11, 2021

I've installed it again the standard way and it has been much easier:

  • pacman -S mingw-w64-x86_64-openssl in msys2 console
  • ensure cabal has the required values in the global config:
extra-include-dirs: D:\path\to\msys64\mingw64\include
extra-lib-dirs: D:\path\to\msys64\mingw64\lib
  • and simply cabal install cabal2nix

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants