Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

use case: cross-compiling to macOS with a dependency on iconv #10485

Open
hzuo opened this issue Jan 1, 2022 · 4 comments
Open

use case: cross-compiling to macOS with a dependency on iconv #10485

hzuo opened this issue Jan 1, 2022 · 4 comments
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. os-macos use case Describes a real use case that is difficult or impossible, but does not propose a solution.
Milestone

Comments

@hzuo
Copy link

hzuo commented Jan 1, 2022

Zig Version

0.9.0

Steps to Reproduce

Importantly, this was working fine on Zig 0.8.1.

On a Linux x86_64 system, with the following environment variables:

export CC="/tmp/zig/zig-cc.sh"
export RUSTFLAGS="-C linker=/tmp/zig/zig-cc.sh" # tells cargo which linker to use
export ZIG_CC_TARGET="x86_64-macos" # see below for zig-cc.sh

/tmp/zig/zig-cc.sh:

#!/usr/bin/env bash

zig cc -target "$ZIG_CC_TARGET" "$@"

Rust has the appropriate toolchain installed as well via rustup target add x86_64-apple-darwin.

Running the following fails:
cargo build --release --target x86_64-apple-darwin

Expected Behavior

Cross-compiling should succeed, as it previously did on 0.8.1.

Actual Behavior

The error message:

#18 25.26           + zig cc -target x86_64-macos -m64 -arch x86_64 /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.0.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.1.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.10.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.11.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.12.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.13.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.14.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.15.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.2.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.3.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.4.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.5.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.6.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.7.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.8.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.dataland_cli.1d4d3fd3-cgu.9.rcgu.o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2.1l6jpj3oj2sa9qhc.rcgu.o -L /workspace/target/x86_64-apple-darwin/release/deps -L /workspace/target/release/deps -L /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib /workspace/target/x86_64-apple-darwin/release/deps/libtracing_subscriber-121def450f384ec8.rlib /workspace/target/x86_64-apple-darwin/release/deps/libtracing_serde-1b1e201a0b2a4537.rlib /workspace/target/x86_64-apple-darwin/release/deps/libserde_json-3b25b60936215023.rlib /workspace/target/x86_64-apple-darwin/release/deps/libryu-ab0a6f0a7a7424d9.rlib /workspace/target/x86_64-apple-darwin/release/deps/libitoa-d2e7789291e807c0.rlib /workspace/target/x86_64-apple-darwin/release/deps/libserde-8ca663f1ba6ddb03.rlib /workspace/target/x86_64-apple-darwin/release/deps/libthread_local-2a1d6d7d5266e0d7.rlib /workspace/target/x86_64-apple-darwin/release/deps/libsharded_slab-4698239f188880b7.rlib /workspace/target/x86_64-apple-darwin/release/deps/libansi_term-a52ae630949588fb.rlib /workspace/target/x86_64-apple-darwin/release/deps/libtracing-3907d690e9347722.rlib /workspace/target/x86_64-apple-darwin/release/deps/libmatchers-56fdde59b4deb082.rlib /workspace/target/x86_64-apple-darwin/release/deps/libregex_automata-f2cefdfc3854ef00.rlib /workspace/target/x86_64-apple-darwin/release/deps/libregex-7cd7c076dda0dab5.rlib /workspace/target/x86_64-apple-darwin/release/deps/libregex_syntax-221c06f8a6a14612.rlib /workspace/target/x86_64-apple-darwin/release/deps/libtracing_log-de77fad8f7d69245.rlib /workspace/target/x86_64-apple-darwin/release/deps/libtracing_core-94d923876d1cd686.rlib /workspace/target/x86_64-apple-darwin/release/deps/libanyhow-8b91ea82dfb23487.rlib /workspace/target/x86_64-apple-darwin/release/deps/libbacktrace-a7571d3ce9bdaf1c.rlib /workspace/target/x86_64-apple-darwin/release/deps/libobject-26177e9e72dd48de.rlib /workspace/target/x86_64-apple-darwin/release/deps/libaddr2line-12ac3d7d499b72d5.rlib /workspace/target/x86_64-apple-darwin/release/deps/libgimli-e783ad841c63807b.rlib /workspace/target/x86_64-apple-darwin/release/deps/librustc_demangle-4264c06b0774ac65.rlib /workspace/target/x86_64-apple-darwin/release/deps/libtokio-a95341b0c068d924.rlib /workspace/target/x86_64-apple-darwin/release/deps/libsignal_hook_registry-e6aca81d26e7d74a.rlib /workspace/target/x86_64-apple-darwin/release/deps/libnum_cpus-c796ac8e638a7923.rlib /workspace/target/x86_64-apple-darwin/release/deps/libmemchr-b2f371f226219eb7.rlib /workspace/target/x86_64-apple-darwin/release/deps/libonce_cell-788000b5d451ef73.rlib /workspace/target/x86_64-apple-darwin/release/deps/libbytes-a4b23a77b0d5639d.rlib /workspace/target/x86_64-apple-darwin/release/deps/libmio-2ea0f919e831f7a1.rlib /workspace/target/x86_64-apple-darwin/release/deps/liblog-57b33a9561401c1c.rlib /workspace/target/x86_64-apple-darwin/release/deps/libpin_project_lite-95c272eb16590914.rlib /workspace/target/x86_64-apple-darwin/release/deps/libparking_lot-66e871389e7cfb65.rlib /workspace/target/x86_64-apple-darwin/release/deps/libparking_lot_core-0e4ee11b437dd388.rlib /workspace/target/x86_64-apple-darwin/release/deps/libsmallvec-2ece64d0efa93d4c.rlib /workspace/target/x86_64-apple-darwin/release/deps/liblock_api-f3d4fdfd0b0c4cda.rlib /workspace/target/x86_64-apple-darwin/release/deps/libscopeguard-3e1eb989dbc64aea.rlib /workspace/target/x86_64-apple-darwin/release/deps/libinstant-db126e86457b2752.rlib /workspace/target/x86_64-apple-darwin/release/deps/libcfg_if-6c282713920cb282.rlib /workspace/target/x86_64-apple-darwin/release/deps/libstructopt-ac4f8199383edc2f.rlib /workspace/target/x86_64-apple-darwin/release/deps/liblazy_static-dd7341373b83f87a.rlib /workspace/target/x86_64-apple-darwin/release/deps/libclap-30139c84668193d2.rlib /workspace/target/x86_64-apple-darwin/release/deps/libvec_map-74a07ca8f456d6ac.rlib /workspace/target/x86_64-apple-darwin/release/deps/libtextwrap-28cd32cd2b0263b2.rlib /workspace/target/x86_64-apple-darwin/release/deps/libunicode_width-13bdbd2bd29a774e.rlib /workspace/target/x86_64-apple-darwin/release/deps/libstrsim-cce460edd015dfaa.rlib /workspace/target/x86_64-apple-darwin/release/deps/libbitflags-b9b1828bf04cff48.rlib /workspace/target/x86_64-apple-darwin/release/deps/libatty-21ad3d79f2f21e38.rlib /workspace/target/x86_64-apple-darwin/release/deps/liblibc-39e24e4029278f6c.rlib /workspace/target/x86_64-apple-darwin/release/deps/libansi_term-9abb6ae0d5d4492f.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libstd-dd8a82589e0cba34.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-8c04c8bd0d1a8900.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libobject-c6a4ae86ed2c40d0.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libmemchr-f9ab4d1b2e38b05e.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libaddr2line-002c7b677ad6c512.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libgimli-a3f3d9f86c37973f.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libstd_detect-8b14bcf2354140fd.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-d6f2fd91ec8bbbcc.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-24c80e37fb5b15c5.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-edb9b11fa36b4795.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libunwind-769780536fb7ef9b.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-d37c37a3a3ac2b0c.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/liblibc-c1bdc4c1f89760ef.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/liballoc-750380e9c94de9ce.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-1108e622f5a15c3d.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libcore-43af7053e70b1eed.rlib /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-3a81ebf6a3abbdee.rlib -liconv -lSystem -lresolv -lc -lm -liconv -L /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib -o /workspace/target/x86_64-apple-darwin/release/deps/dataland_cli-c446093d9b895ef2 -Wl,-dead_strip -nodefaultlibs
#18 25.26           warning: unsupported linker arg: -dead_strip
#18 25.26           warning(link): library not found for '-liconv'
#18 25.26           warning(link): Library search paths:
#18 25.26           warning(link):   /workspace/target/x86_64-apple-darwin/release/deps
#18 25.26           warning(link):   /workspace/target/release/deps
#18 25.26           warning(link):   /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib
#18 25.26           warning(link):   /usr/local/rustup/toolchains/1.57.0-x86_64-unknown-linux-gnu/lib/rustlib/x86_64-apple-darwin/lib
#18 25.26           error: LibraryNotFound

Seems that 0.9.0 can't find iconv but 0.8.1 was able to?

@hzuo hzuo added the bug Observed behavior contradicts documented or intended behavior label Jan 1, 2022
@hzuo hzuo changed the title [0.9.0 Regression] Linking fails on -dead_strip -liconv flags when cross-compiling Rust to MacOS [0.9.0 Regression] Linking fails to find iconv library when cross-compiling Rust to MacOS Jan 1, 2022
@hzuo
Copy link
Author

hzuo commented Jan 1, 2022

Seems related to #9725

@andrewrk andrewrk added the regression It worked in a previous version of Zig, but stopped working. label Jan 1, 2022
@andrewrk andrewrk added this to the 0.9.1 milestone Jan 1, 2022
@kubkon
Copy link
Member

kubkon commented Jan 11, 2022

Since you're invoking Zig with -target flag, it enters cross-compilation mode so you will need to explicitly pass the sysroot and specify the search paths for include dirs, libraries and frameworks like so:

#!/usr/bin/env bash

zig cc -target "$ZIG_CC_TARGET" --sysroot=/path/to/sysroot -I/usr/include -L/usr/lib -F/System/Library/Frameworks "$@"

Note that this is also conditional on this fix #10568

@kubkon
Copy link
Member

kubkon commented Jan 14, 2022

@hzuo OK, I seem to have misunderstood the issue, however, it is not a regression per se. Let me explain. The behaviour (and error) are actually correct. -liconv is required by Rust (as evidenced by its presence on the linker line) and it's available via macOS SDK, therefore you will need to provide the sysroot/SDK when cross-compiling Rust to macOS from a foreign host. The reason this was not an error before was because zig ld did not flag the missing library as error but rather as warning, and would link away. This is the only change in behaviour since 0.8. Also, FYI, I did provide the sysroot/SDK, and cross-linked basic hyper.rs hello-world example just fine.

@andrewrk do you think we should leave it at status quo of this being a hard error? I for one stand by this and believe it should always be a hard error.

@andrewrk andrewrk removed bug Observed behavior contradicts documented or intended behavior regression It worked in a previous version of Zig, but stopped working. labels Jan 15, 2022
@andrewrk andrewrk modified the milestones: 0.9.1, 0.10.0 Jan 15, 2022
@andrewrk
Copy link
Member

Yes if you specify to link a library with -liconv then it must link iconv or report an error.

As for how to solve @hzuo's problem, they will need to provide an iconv to link against when cross compiling, or if possible, pass a build configuration option telling it to not depend on iconv.

If you wish to use zig cc to cross compile and you want to avoid a macOS SDK, I suggest to first cross-compile iconv with zig, then you will have a static iconv build that you can link against.

I will leave this open for a while as a "use case" to remind us that solving this problem somehow could help Rust folks - but there is a good chance that it will be eventually closed with no solution.

@andrewrk andrewrk added enhancement Solving this issue will likely involve adding new logic or components to the codebase. use case Describes a real use case that is difficult or impossible, but does not propose a solution. labels Jan 15, 2022
@andrewrk andrewrk modified the milestones: 0.10.0, 0.11.0 Jan 15, 2022
@andrewrk andrewrk changed the title [0.9.0 Regression] Linking fails to find iconv library when cross-compiling Rust to MacOS use case: cross-compiling to macOS with a dependency on iconv Jan 15, 2022
luislavena added a commit to luislavena/crystal that referenced this issue Mar 5, 2022
Allow developers to opt for libiconv variant instead of the built-in
iconv support of their platform.

This helps with cross-compilation of binaries to platforms like macOS,
which requires bulky SDK being installed in order to support linking to
their bundled `iconv` library.

Alternatively, developer should be able to build a static version of
iconv (`libiconv`) and be able to link with it by forcing its usage with
`-Duse_libiconv`:

  $ crystal build app.cr \
      --cross-compile --target x86_64-apple-darwin -Duse_libiconv

Relevant links:
- https://forum.crystal-lang.org/t/macos-compilation-and-libiconv-support/4439
- ziglang/zig#10485 (comment)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement Solving this issue will likely involve adding new logic or components to the codebase. os-macos use case Describes a real use case that is difficult or impossible, but does not propose a solution.
Projects
None yet
Development

No branches or pull requests

3 participants