How to link a static lib (zlib) using cli options instead extra-libraries #5805

jneira · 2018-12-17T22:20:13Z

Hi, i've tried to build the package hopenssl, including the test suite in windows. The steps i've followed were:

download openssl libs from https://sourceforge.net/projects/openssl/files/openssl-1.0.2j-fips-x86_64/
setting appropiate cabal global config options:

extra-include-dirs: X:\path\to\OpenSSL-x86_64\include
extra-lib-dirs: X:\path\to\OpenSSL-x86_64\lib

Download the lib with cabal get hopenssl-2.2.2
executing cabal new-build --enable-tests -v3

At first the build failed linking one of the executables tests:

 Linking C:\Users\Javier\dev\lang\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\t\check-low-level-digest-api\build\check-low-level-digest-api\check-low-level-digest-api.exe ...
C:\Users\Javier\dev\lib\openssl-1.0.2j-fips-x86_64\lib/libcrypto.a(c_zlib.o):c_zlib.c:(.text+0x59): undefined reference to `inflate'
C:\Users\Javier\dev\lib\openssl-1.0.2j-fips-x86_64\lib/libcrypto.a(c_zlib.o):c_zlib.c:(.text+0xea): undefined reference to  `deflate'
.... lot of undefined refs of zlib.c ...
collect2.exe: error: ld returned 1 exit status
`gcc.exe' failed in phase `Linker'. (Exit code: 1)
cabal: Failed to build test:check-high-level-digest-api from hopenssl-2.2.2.
Failed to build test:check-low-level-digest-api from hopenssl-2.2.2.

Then i tested to add the z lib in the extra-libraries section of the cabal file and it worked, without pointing to zlib location explicitly (full log).

However it seems the package shouldn't add the lib, cause it is a specific issue of the concrete openssl lib implementation used in this case.
So i thought the solution could be to add the -lz option cause in the succesful case i detected it was being passed to hsc2hs and gcc (but no to ghc, at least explicitly). However i had not luck passing -lz in --gcc-options, --ghc-options, --ld-options, etc. (full log of cabal new-build --enable-tests --gcc-options=-lz)

Note: The build of hackage2nix executable of cabal2nix package fails with the same error. That is my primary goal (see NixOS/cabal2nix#389)
Thanks in advance!

The text was updated successfully, but these errors were encountered:

hvr · 2018-12-17T23:11:16Z

Are you experiencing #2997 ?

If yes, adding the following to your cabal.project file is the currently known workaround:

-- ...

package hopenssl
   extra-lib-dirs: X:\path\to\OpenSSL-x86_64\include
   extra-include-dirs: X:\path\to\OpenSSL-x86_64\lib

jneira · 2018-12-18T05:37:27Z

Thanks, i think it is not the cause: if i remove extra-lib-dirs and extra-include-libs from global ~\cabal\config i got errors in the lib build about missing crypto headers or the lib itself. Afaiu (not much!) it is the crypto c lib impl itself which needs exports from z but doesn't link it statically.
So using the extra-libraries: crypto z in hopenssl.cabal section makes ghc(?) link z lib as well but i've not able to found a way to replicate the behaviour using cli options (or cabal.project package specific options)

jneira · 2018-12-18T07:06:46Z

I should note that cabal v1-build fails with the same error.
To help identify the cause i've extracted the two calls from verbose output that have explicit references to -lz:

Succesful build (with extra-libraries: crypto z in hopenssl.cabal lib section and cabal new-build --enable-tests -v3):

First call to gcc

"D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
	"-fno-stack-protector" "C:\TEMP\9232-4.c" 
	"-o" "C:\TEMP\9232-5" 
	"-D__GLASGOW_HASKELL__=804" "-Dmingw32_BUILD_OS=1" "-Dx86_64_BUILD_ARCH=1" "-Dmingw32_HOST_OS=1" "-Dx86_64_HOST_ARCH=1" 
	"-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen" 
	"-ID:\dev\OpenSSL-x86_64\include" 
	"-I." 
	"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2\include" 
	"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0\include" 
	"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0\include" 
	"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/include" 
	"-lcrypto" "-lz" 
	"-LD:\dev\OpenSSL-x86_64\lib" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\deepseq-1.4.3.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\array-0.5.2.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\ghc-prim-0.5.2.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/rts"

First and unique call to hsc2hs

"D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\bin\hsc2hs.exe" 
	"--cc=D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
	"--ld=D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
	"--cflag=-fno-stack-protector" "--lflag=-fno-stack-protector" 
	"--cflag=-D__GLASGOW_HASKELL__=804" "--cflag=-Dmingw32_BUILD_OS=1" "--cflag=-Dx86_64_BUILD_ARCH=1" "--cflag=-Dmingw32_HOST_OS=1" "--cflag=-Dx86_64_HOST_ARCH=1" 
	"--cflag=-ID:\dev\OpenSSL-x86_64\include" 
	"--cflag=-ID:\dev\OpenSSL-x86_64\include" 
	"--cflag=-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen" 
	"--cflag=-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\global-autogen" 
	"--cflag=-include" "--cflag=D:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen\cabal_macros.h" 
	"--lflag=-LD:\dev\OpenSSL-x86_64\lib" 
	"--lflag=-lcrypto" "--lflag=-lz" 
	"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2\include" 
	"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0\include" 
	"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0\include" 
	"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/include" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\deepseq-1.4.3.0" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\array-0.5.2.0" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0" 
	"--lflag=-lwsock32" "--lflag=-luser32" "--lflag=-lshell32" "--lflag=-lmsvcrt" "--lflag=-lmingw32" "--lflag=-lmingwex" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\ghc-prim-0.5.2.0" 
	"--lflag=-luser32" "--lflag=-lmingw32" "--lflag=-lmingwex" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/rts" 
	"--lflag=-lm" "--lflag=-lwsock32" "--lflag=-lgdi32" "--lflag=-lwinmm" "--lflag=-lDbghelp" "--lflag=-lPsapi" 
	"-o" "D:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\OpenSSL\EVP\Digest\Algorithm.hs" "src\OpenSSL\EVP\Digest\Algorithm.hsc"

Wrong build (hopenssl.cabal original and cabal new-build --enable-tests -v3 --gcc-options=-lz)

First call to gcc

"D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
	"-fno-stack-protector" "C:\TEMP\9692-4.c" 
	"-o" "C:\TEMP\9692-5" 
	"-D__GLASGOW_HASKELL__=804" "-Dmingw32_BUILD_OS=1" "-Dx86_64_BUILD_ARCH=1" "-Dmingw32_HOST_OS=1" "-Dx86_64_HOST_ARCH=1" 
	"-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen" 
	"-ID:\dev\OpenSSL-x86_64\include" 
	"-I." 
	"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2\include" 
	"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0\include" 
	"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0\include" 
	"-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/include" 
	"-lcrypto" 
	"-LD:\dev\OpenSSL-x86_64\lib" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\deepseq-1.4.3.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\array-0.5.2.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\ghc-prim-0.5.2.0" 
	"-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/rts" 
	"-lz" "-lz"

First and unique call to hsc2hs:

"D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\bin\hsc2hs.exe" 
	"--cc=D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
	"--ld=D:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/../mingw/bin\gcc.exe" 
	"--cflag=-fno-stack-protector" 
	"--cflag=-lz" "--cflag=-lz" 
	"--lflag=-fno-stack-protector" 
	"--lflag=-lz" "--lflag=-lz" 
	"--cflag=-D__GLASGOW_HASKELL__=804" "--cflag=-Dmingw32_BUILD_OS=1" "--cflag=-Dx86_64_BUILD_ARCH=1" "--cflag=-Dmingw32_HOST_OS=1" "--cflag=-Dx86_64_HOST_ARCH=1" 
	"--cflag=-ID:\dev\OpenSSL-x86_64\include" 
	"--cflag=-ID:\dev\OpenSSL-x86_64\include" 
	"--cflag=-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen" 
	"--cflag=-ID:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\global-autogen" 
	"--cflag=-include" 
	"--cflag=D:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\autogen\cabal_macros.h" 
	"--lflag=-LD:\dev\OpenSSL-x86_64\lib" 
	"--lflag=-lcrypto" 
	"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2\include" 
	"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0\include" 
	"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0\include" 
	"--cflag=-ID:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/include" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\bytestring-0.10.8.2" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\deepseq-1.4.3.0" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\array-0.5.2.0" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\base-4.11.1.0" 
	"--lflag=-lwsock32" "--lflag=-luser32" "--lflag=-lshell32" "--lflag=-lmsvcrt" "--lflag=-lmingw32" "--lflag=-lmingwex" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\integer-gmp-1.0.2.0" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib\ghc-prim-0.5.2.0" 
	"--lflag=-luser32" "--lflag=-lmingw32" "--lflag=-lmingwex" 
	"--lflag=-LD:\dev\bin\Programs\stack\x86_64-windows\ghc-8.4.4\lib/rts" 
	"--lflag=-lm" "--lflag=-lwsock32" "--lflag=-lgdi32" "--lflag=-lwinmm" "--lflag=-lDbghelp" "--lflag=-lPsapi" 
	"-o" "D:\ws\eta\dhall\hopenssl-2.2.2\dist-newstyle\build\x86_64-windows\ghc-8.4.4\hopenssl-2.2.2\build\OpenSSL\EVP\Digest\Algorithm.hs" "src\OpenSSL\EVP\Digest\Algorithm.hsc"

I've noticed those points:

There is no calls to programs (including ghc) with -lz explicitly passed as flag other than the above ones (in both builds)
The diffs between them are:
- The position of -lcrypto and lz are different (although the relative position between them is the same: first crypto and second z)
- The -lz flag is being duplicated when using --gcc-options=-lz

jneira · 2018-12-18T08:49:38Z

@hvr Would be a possible fix that cabal.project allow replace or add more extra-libraries to a package?:

package hopenssl
        extra-libraries: crypto z

or

package hopenssl
        add-extra-libraries: z

Not sure if it shoud be a per-component ...

However it should be a way to link z with cli options, right?

hvr · 2018-12-18T09:40:52Z

@jneira is this a dynamic library? if so, transitive inter-library dependencies should be resolved automatically when libssl internally needs e.g. zlib

have you tried injecting -lz via something like

package hopenssl
        ghc-options: -lz

(in principle we could add support for injecting extra-libraries -- I'm just not sure if this is a common enough problem to justify the implementation complexity, over just letting you fix the issue upstream (e.g. upstream could either add autoconf tests detecting the need for -lz or provide a cabal flag for enabling an extra-libraries: z) or vendoring the package)

I'm a bit in a hurry, so please excuse me if I have missed something in your comments which makes my suggestion moot. I'll try to reread more carefully when I have more time later today.

jneira · 2018-12-18T10:13:13Z

@hvr Dont worry and let me know if i can collect/extract more info to make easier the diagnostic.

have you tried injecting -lz via something like
package hopenssl
        ghc-options: -lz

I am afraid that i've tried it. Due to my ignorance about build internals i've tried adding to gcc-options, ld-options and hsc2hs-options too. In the last case using --lflag=-lz.
I've experimented passing -optl- options in ghc-options to pass params directly to linker... with no luck.

I'm just not sure if this is a common enough problem to justify the implementation complexity

Agree

over just letting you fix the issue upstream

Package maintainer argues that the root problem is in the crypto c library implementation chosen in this case and include z should not be responsability of the package. It seems reasonable.

I'll try to reread more carefully when I have more time later today.

The main conclusion of my analysis comparing the succesful and wrong builds is in #5805 (comment)

Thanks a lot!

Static variants of libcrypto may depend on libz but can't express that dependency, so we have to worry about telling Cabal about it. Relevant Issue: - NixOS/cabal2nix#389 - haskell/cabal#5805

jneira · 2019-02-14T05:30:00Z

I wonder if #5446 would fix this issue too...

jneira · 2019-02-14T05:34:32Z

And maybe the cause of no possible workaround with ld-options is due to #4925? (we should put in the cabal file to test it)
There is a pr about: #5451

nh2 · 2019-08-09T23:06:32Z

And maybe the cause of no possible workaround with ld-options is due to #4925?

This is now fixed with the merged PR #5451.

jneira · 2019-08-09T23:25:12Z

Ok, i am trying again and i have the two calls of ghc linker phase:

With z as extra-library that works:

"D:\bin\stack\x86_64-windows\ghc-8.6.5\lib\../mingw/bin/gcc.exe" "-fno-stack-protector" \ 
"-DTABLES_NEXT_TO_CODE" "-Wl,--hash-size=31" "-Wl,--reduce-memory-overheads" \ 
"-fstack-check" "-static-libgcc" "-v" "-v" "-o" \ 
"D:\dev\ws\haskell\hopenssl-2.2.4\dist-newstyle\build\x86_64-windows\ghc-8.6.5\hopenssl-2.2.4\t\check-high-level-digest-api\build\check-high-level-digest-api\check-high-level-digest-api.exe" \ 
"-lm" "-no-pie" "-Wl,--enable-auto-import" "-Wl,--gc-sections" \ 
"D:\dev\ws\haskell\hopenssl-2.2.4\dist-newstyle\build\x86_64-windows\ghc-8.6.5\hopenssl-2.2.4\t\check-high-level-digest-api\build\check-high-level-digest-api\check-high-level-digest-api-tmp\OpenSesame.o" \ 
"D:\dev\ws\haskell\hopenssl-2.2.4\dist-newstyle\build\x86_64-windows\ghc-8.6.5\hopenssl-2.2.4\t\check-high-level-digest-api\build\check-high-level-digest-api\check-high-level-digest-api-tmp\Main.o" \ 
"-LD:\dev\lib\openssl-1.0.2j-fips-x86_64\lib" "C:\Users\Javier\AppData\Local\Temp\ghc4948_0\ghc_5.o" \ 
"-LD:\dev\ws\haskell\hopenssl-2.2.4\dist-newstyle\build\x86_64-windows\ghc-8.6.5\hopenssl-2.2.4\build" \ 
"-LD:\dev\lib\openssl-1.0.2j-fips-x86_64\lib" "-LD:\bin\stack\x86_64-windows\ghc-8.6.5\lib\bytestring-0.10.8.2" (other -Lpackages) \
 "C:\Users\Javier\AppData\Local\Temp\ghc4948_0\ghc_2.o" "-Wl,-u,base_GHCziTopHandler_runIO_closure"  (more -Wl,u,xxx) \ 
"-lHShopenssl-2.2.4-inplace" (other -lpackages) "-lHSrts_thr" "-lCffi-6" \ 
"-lcrypto" "-lz" "-lwsock32" "-luser32" "-lshell32" "-lmsvcrt" "-lmingw32" \ 
"-lmingwex" "-luser32" "-lmingw32" "-lmingwex" "-lm" "-lwsock32" "-lgdi32" \ 
"-lwinmm" "-ldbghelp" "-lpsapi"

with: cabal v2-build --enable-tests -v3 --ghc-options=-v3 --ld-options="-LD:\dev\lib\openssl-1.0.2j-fips-x86_64\lib -LD:\path\to\ghc-8.6.5\mingw\lib -lcrypto -lz -v" (after Pass ld-options through to GHC #5451 -ld-options are passed to ghc and it is no need to --ghc-options=-optl-xxx) that fails

"D:\bin\stack\x86_64-windows\ghc-8.6.5\lib\../mingw/bin/gcc.exe" "-fno-stack-protector"\
 "-DTABLES_NEXT_TO_CODE" "-Wl,--hash-size=31" "-Wl,--reduce-memory-overheads" \ 
"-fstack-check" "-static-libgcc" \ 
"-LD:\dev\lib\openssl-1.0.2j-fips-x86_64\lib" \
"-LD:\bin\stack\x86_64-windows\ghc-8.6.5\mingw\lib" \
"-lcrypto" "-lz" "-v" \
"-LD:\dev\lib\openssl-1.0.2j-fips-x86_64\lib" \
"-LD:\bin\stack\x86_64-windows\ghc-8.6.5\mingw\lib" \ 
"-lcrypto" "-lz" "-v" "-o" \ 
"D:\dev\ws\haskell\hopenssl-2.2.4\dist-newstyle\build\x86_64-windows\ghc-8.6.5\hopenssl-2.2.4\t\check-high-level-digest-api\build\check-high-level-digest-api\check-high-level-digest-api.exe" \ 
"-lm" "-no-pie" "-Wl,--enable-auto-import" "-Wl,--gc-sections" \
"D:\dev\ws\haskell\hopenssl-2.2.4\dist-newstyle\build\x86_64-windows\ghc-8.6.5\hopenssl-2.2.4\t\check-high-level-digest-api\build\check-high-level-digest-api\check-high-level-digest-api-tmp\OpenSesame.o" \
"D:\dev\ws\haskell\hopenssl-2.2.4\dist-newstyle\build\x86_64-windows\ghc-8.6.5\hopenssl-2.2.4\t\check-high-level-digest-api\build\check-high-level-digest-api\check-high-level-digest-api-tmp\Main.o" \
"-LD:\dev\lib\openssl-1.0.2j-fips-x86_64\lib" "C:\Users\Javier\AppData\Local\Temp\ghc10700_0\ghc_5.o" \
"-LD:\dev\ws\haskell\hopenssl-2.2.4\dist-newstyle\build\x86_64-windows\ghc-8.6.5\hopenssl-2.2.4\build" \
"-LD:\dev\lib\openssl-1.0.2j-fips-x86_64\lib" \
"-LD:\bin\stack\x86_64-windows\ghc-8.6.5\lib\bytestring-0.10.8.2" (more -Lpackages)  "C:\Users\Javier\AppData\Local\Temp\ghc10700_0\ghc_2.o" "-Wl,-u,base_GHCziTopHandler_runIO_closure" (more -Wl,-u,xxxx) \
"-lHShopenssl-2.2.4-inplace" "-lHSbytestring-0.10.8.2" (more -lpackages)  \
"-lHSrts_thr" "-lCffi-6" "-lcrypto" "-lwsock32" "-luser32" "-lshell32" "-lmsvcrt" \
 "-lmingw32" "-lmingwex" "-luser32" "-lmingw32" "-lmingwex" "-lm" "-lwsock32" \
"-lgdi32" "-lwinmm" "-ldbghelp" "-lpsapi"

AFAIU the main difference is the position where -lz is placed: in the correct execution it is at the end with other default libraries, after object files.
In the incorrect one it is placed at the beginning, before any other library or object file. It seems the order can be significant in the resolution of definitions so maybe it is the cause of the problem.

jneira · 2019-08-09T23:33:41Z

This is now fixed with the merged PR #5451.

@nh2 thanks for the patch! i am afraid that the cause is in the order of linker options (not sure).

nh2 · 2019-08-09T23:39:26Z

@jneira This may be the same problem as I had here:

The flags are passed to the linker, but they appear too early in the cc invocation; I intended them to appear at the very end but they appear before -o and before all the Haskell libraries.

In my case I think ghc puts the flags in the wrong position, but maybe I'm wrong given that it works fine when extra-libraries is used instead.

jneira · 2019-08-09T23:58:49Z

@nh2 wow, thanks for pointed out your insightful comments about the linker.
I had read about --start-group ld option but i had not tried it cause i didnt see the way to specify --end-group, but as you noted, it works without closing it!

So now i've got to compile the package with
cabal v2-build --enable-tests --ld-options="-Wl,--start-group -LD:\dev\lib\openssl-1.0.2j-fips-x86_64\lib -LD:\bin\stack\x86_64-windows\ghc-8.6.5\mingw\lib -lz -lcrypto"

I know it is somewhat a hack (as you noted too) but i think it is difficult to tell cabal the position of prog-args (why only ld ones?), so i am gonna close this one.
Nevertheless it would be great to be able to use prog-options in a more fine grained way.

nh2 · 2019-08-10T00:10:21Z

@jneira I think we'll have to fix this in ghc. Would you mind filing a bug for this issue there and CC'ing me there? I'm pretty swamped with issues currently :)

jneira · 2019-08-13T21:22:35Z

@nh2 i am doing a little bit of investigation: it is clear that ghc supports the cabal use of extra-libraries, putting them at the end of linker call, its correct place. But i am not able to determine how cabal makes ghc do it that way.
Afaiu the "semantic" layout of linker params is:

General default params
User defined params from optl
Executable name and more generic params
Object files
List of undefined references to be ignored with -Wl,-u,...
Packages
Extra libraries

When you inform the extra-libraries cabal field it is able to make ghc puts them in the last section and for me it is a good semantic way to do it.
My original problem was you can't tell cabal to add those extra libraries to external packages through cabal.project or the cli but i think it would be the best way to fix that one.

nh2 · 2019-09-09T07:20:10Z

I think we'll have to fix this in ghc

@jneira I have done this now: https://gitlab.haskell.org/ghc/ghc/issues/17071

I've also made a PR to fix the problem. It moves optl from your mentioned position 2 to the very end.

jneira · 2019-09-09T07:29:22Z

@nh2 great! It seems that is the good place to put them.
Anyway i am a little bit intrigued why extra-libraries went correctly in last position in the first place.

nh2 · 2019-09-09T08:18:51Z

I think it's because they are passed as -l flags directly to ghc. On the cabal side:

cabal/Cabal/Distribution/PackageDescription/FieldGrammar.hs

Line 421 in 63331c9

    
           <*> monoidalFieldAla "extra-libraries"      (alaList' VCat Token)         L.extraLibs

extra-libraries is parsed into extraLibs. Then

cabal/Cabal/Distribution/Simple/GHC.hs

Line 847 in 63331c9

ghcOptLinkLibs = extraLibs libBi,

it's called ghcOptLinkLibs = extraLibs libBi from which the GHC CLI arguments are rendered in

cabal/Cabal/Distribution/Simple/Program/GHC.hs

Lines 670 to 674 in 63331c9

    
           -- Linker stuff 
        
           , [ "-optl" ++ opt | opt <- ghcOptLinkOptions opts] 
        
           , ["-l" ++ lib     | lib <- ghcOptLinkLibs opts] 
        
           , ["-L" ++ dir     | dir <- flags ghcOptLinkLibPath ]

where we can see that with ["-l" ++ lib | lib <- ghcOptLinkLibs opts] it's passed as -l directly to GHC.

On the GHC side (link) those probably get translated to be the last contents of args given to runLink.

I haven't checked in detail yet where e.g. your points (6) and (7) are joined into args inside GHC.

jneira mentioned this issue Dec 17, 2018

Build error of hackage2nix executable in windows with openssl-1.0.2j-fips-x86_64 NixOS/cabal2nix#389

Closed

jneira mentioned this issue Aug 9, 2019

--prog-option(s) (e.g. gcc-options) CLI flags get duplicated #6183

Open

jneira closed this as completed Aug 9, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

How to link a static lib (zlib) using cli options instead extra-libraries #5805

How to link a static lib (zlib) using cli options instead extra-libraries #5805

jneira commented Dec 17, 2018 •

edited

Loading

hvr commented Dec 17, 2018

jneira commented Dec 18, 2018 •

edited

Loading

jneira commented Dec 18, 2018 •

edited

Loading

jneira commented Dec 18, 2018 •

edited

Loading

hvr commented Dec 18, 2018 •

edited

Loading

jneira commented Dec 18, 2018 •

edited

Loading

jneira commented Feb 14, 2019

jneira commented Feb 14, 2019 •

edited

Loading

nh2 commented Aug 9, 2019

jneira commented Aug 9, 2019 •

edited

Loading

jneira commented Aug 9, 2019

nh2 commented Aug 9, 2019

jneira commented Aug 9, 2019

nh2 commented Aug 10, 2019

jneira commented Aug 13, 2019

nh2 commented Sep 9, 2019

jneira commented Sep 9, 2019

nh2 commented Sep 9, 2019

How to link a static lib (zlib) using cli options instead extra-libraries #5805

How to link a static lib (zlib) using cli options instead extra-libraries #5805

Comments

jneira commented Dec 17, 2018 • edited Loading

hvr commented Dec 17, 2018

jneira commented Dec 18, 2018 • edited Loading

jneira commented Dec 18, 2018 • edited Loading

jneira commented Dec 18, 2018 • edited Loading

hvr commented Dec 18, 2018 • edited Loading

jneira commented Dec 18, 2018 • edited Loading

jneira commented Feb 14, 2019

jneira commented Feb 14, 2019 • edited Loading

nh2 commented Aug 9, 2019

jneira commented Aug 9, 2019 • edited Loading

jneira commented Aug 9, 2019

nh2 commented Aug 9, 2019

jneira commented Aug 9, 2019

nh2 commented Aug 10, 2019

jneira commented Aug 13, 2019

nh2 commented Sep 9, 2019

jneira commented Sep 9, 2019

nh2 commented Sep 9, 2019

jneira commented Dec 17, 2018 •

edited

Loading

jneira commented Dec 18, 2018 •

edited

Loading

jneira commented Dec 18, 2018 •

edited

Loading

jneira commented Dec 18, 2018 •

edited

Loading

hvr commented Dec 18, 2018 •

edited

Loading

jneira commented Dec 18, 2018 •

edited

Loading

jneira commented Feb 14, 2019 •

edited

Loading

jneira commented Aug 9, 2019 •

edited

Loading